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Abstract 

We provide a large deviations analysis of deadlock phenomena occurring in dis- 
tributed systems sharing common resources. In our model transition probabilities 
of resource allocation and deallocation are time and space dependent. The process 
is driven by an ergodic Markov chain and is reflected on the boundary of the d- 
dimensional cube. In the large resource limit, we prove Preidlin-Wentzell estimates, 
we study the asymptotic of the deadlock time and we show that the quasi-potential 
is a viscosity solution of a Hamilton-Jacobi equation with a Neumann boundary 
condition. We give a complete analysis of the colliding 2-stacks problem and show 
an example where the system has a stable attractor which is a limit cycle. 
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1 Introduction 

Distributed algorithms are related to resource sharing problems. Colliding stacks prob- 
lems and the banker algorithm are among the examples which have attracted large in- 
terest over the last decades in the context of deadlock prevention on multiprocessor 
systems. Knuth [22], Yao [32], Flajolet [15], Louchard, Schott et al. [261 E3 [28] have 
provided combinatorial or probabilistic analysis of these algorithms in the 2-dimensional 
case under the assumption that transition probabilities (of allocation or deallocation) are 
constant. Maier [21] proposed a large deviations analysis of colliding stacks for the more 
difficult case where the transition probabilities are non-trivially state-dependent. More 
recently Guillotin-Plantard and Schott [T7J [18] analyzed a model of exhaustion of shared 
resources where allocation and deallocation requests are modeled by time-dependent dy- 
namic random walks. In [8], the present authors provided a probabilistic analysis of 
the o?-dimensional banker algorithm when transition probabilities evolve, as time goes 
by, along the trajectory of an ergodic Markovian environment, whereas the spatial pa- 
rameter just acts on long runs. The analysis in [8] relies on techniques from stochastic 
homogenization theory. In this paper, we consider a similar dynamics, but in a stable 
regime instead of a neutral regime as in our previous paper, and we provide an orig- 
inal large deviations analysis in the framework of Freidlin-Wentzell theory. Given the 
environment, the process of interest is a Markov process depending on the number m 
of available resource, with smaller and more frequent jumps as m — > oo, see (13.11) . A 
number of monographs and papers have been written on this theory: [14] . [T6] and [19] 
for random environment, [ID] . [12], [21] and [20] for reflected processes, [3], [9] and [30] for 
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homogeneous Markov processes. However, our framework, including both reflections on 
the boundary and averaging on the Markovian environment, is not covered by the current 
literature, and we establish here the large deviations principle. We prove that the time 
of resource exhaustion then grows exponentially with the size of the system - instead of 
polynomially in the neutral regime of [H] and has exponential law as limit distribution. 
Then, we study the quasi-potential, which solves, according to general wisdom, some 
Hamilton- Jacobi equation: in view of the reflection on the hypercube, which boundary 
is non-regular, we prove this fact in the framework of viscosity solution, and study the 
optimal paths (so-called instantons). 
We investigate in details a particular situation introduced in a beautiful paper of Maier 
where the motion in each direction depends on the corresponding coordinate only, 
with the additional dependence in the Markovian environment. In fact, we discover the 
quasi-potential by observing that the discrete process has an invariant measure, for which 
we study the large deviations properties. We can then use the characterization in terms 
of Hamilton- Jacobi equation to bypass the Hamiltonian mechanics approach of [29J . For 
the deadlock phenomenon, we finally obtain a (even more) complete picture (after an 
even shorter work). To the best of our knowledge, this is first such analysis developed 
for space-time inhomogeneous distributed algorithms. 

The organization of this paper is as follows: we discuss our probabilistic model in 
Section 2. In Section 3 we prove a Large Deviations Principle. Deadlock phenomenon 
analysis is done rigorously with much details in Section 4. In Section 5 we illustrate with 
the two-stacks model. In Section 6 we work out an example where the system has (in 
the large scale resource limit m — > oo) a stable attractor which is a limit cycle. Some 
technical proofs of results stated in Section 4 are deferred to Appendix (Section 7). 

2 The Model 

The environment is given by a Markov chain (£ n ) n >o defined on (Q, A, P) with values in 
a finite space E, N = \E\. We denote by P its transition matrix, P(k,£) = P(£ n+ i = 
£\£ n = k) for k,£ £ E. 
The steps of the walker take place in the set 

V = {e 1; -ei, ...,e d , -e d } , 

where (ej)i<j<d denotes the canonical basis of Z d , and are reflected along the boundary 
of the hypercube [0,m] d , for a large integer m. 

Following [8J, we first discuss the dynamics of the walk in the non-reflected setting. 
The displacement of the walker has then law (p(s/m, i, v); v £ V), when located at s and 
when the environment is i. To obtain a stochastic representation - which is, in contrast 
to [H], needed here - , we are also given, on (fl,A,F), a sequence (U n ) n > of independent 
and uniformly distributed random variables on (0, 1), independent of the family (£„)„>o- 
Denoting by / : (s,i,v) eR d x Ex (0,1) — > V the inverse of the cumulative distribution 
function of (p(s, i, ■u)) ue y (f° r an arbitrary order on V), we have 

P{/(s, i, U n ) = v} = p(s, i, v) , 



4 



COMETS, DELARUE AND SCHOTT 



so that the position of the walker can be defined recursively by 

S n+ i = S n + f(S n /m,£ n , U n ). (2.1) 

The reflected walk is obtained by symmetry with respect to the faces of the hypercube. 
Denoting by Id the identity mapping on W 1 and by II (resp. lT m )) the projection on the 
hypercube [0, l] d (resp. [0,m] d ), we define recursively the position of the walker by 

X n+1 = (2U^ - Id) (X n + f(X n /m, £ n , U n )) 

= m(2II - Id) (X n /m + (l/m)f{X n /m, £ n , U n )) . 

When X n is on the boundary and X n + f(X n /m, £„, U n ) is outside the hypercube, X n+ i 
is the symmetric point of X n + f(X n , £ n , U n ) with respect to the face containing X n and 
orthogonal to f(X n /m,£ n ,U n ), i.e. X n+1 = X n - f(X n /m,£, n ,U n ). The kernel q of the 
walk (X n ) n >! has the following form. When located at x G (0,m) D Z d and when the 
environment is i, the jump of the walker has law (q(x/m, i, v) = p(x/m, i, v); v G V). On 
the boundary, the reflection rules may be expressed as follows: if xg/m = 1 (xg is the £th 
coordinate of x), q(x/m, i, eg) = and q(x/m, i, —ee) = p(x/m, i, eg) + p(x/m, i, — eg) ; if 
xg = 0, q(x/m, i, —eg) = and q(x/m, i, —eg) = p(x/m, i, eg) + p(x/m, i, —eg). 

We could choose another reflection rule by setting X n+ i = (X n + f (X n / m, £ n , U n )). 
Such a choice wouldn't change anything to the proofs given in the paper, except the proof 
of Theorem 14.91 which uses the fact that the steps of (X n ) n > are always non-zero. 

Following (12. ip . we can write 

X n+1 = X n + g(X n /m, £ n , U n ) , (2.3) 

where g : (x, i, v ) G M. d x E x (0, 1) — > V is the inverse of the cumulative distribution func- 
tion of (q(x, i, u)) u& v. Of course, g(x,i,v) = f(x,i,v) for x G (0, l) d . On the boundary, 
xg = 1 =3- ge(x, i, v ) < and xg = =>- gg(x, i, v) > 0. 
The process (£ n; ^n)n>o is a Markov chain with transition probabilities 

F{^ n+1 = k,X n+1 = v + X n \^ x } = P^ n ,k)q{X n /m^ n ,v), 

where T^ x = cr{^ , . . . , £ n , X , . . . , X n }. In particular, for x G Z d and for a probability 
measure v on E, we can write P^/ m to indicate that the chain starts under the measure 
5 X ® v. In many cases, we just write P x / m (resp. F u ): this means that the law of the 
environment (resp. of the walker) is arbitrary. And, of course, the notation P means that 
both the initial conditions of the walker and of the environment are arbitrary. 

2.1 Main Assumptions 

In the whole paper, (-, •) and \ ■ \ stand for the Euclidean scalar product and the Euclidean 
norm in M. d . The symbols | ■ |i and | • |oo denote the standard i 1 and £°° norms in ~R d . 

From a purely practical point of view, the values ofp(x,i,v) forx outside the hypercube 
[0, l] d are totally useless. In the sequel, we refer, for pedagogical reasons, to the non- 
reflected walk: in such cases, we need p(x,i,v) to be defined for all x G M d . This is the 
reason why the variable x lies in M. d in the following assumptions. 
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In formulas ( 12. lft and ( 12. 2ft . the division by m indicates that the dependence of the 
transition kernel on the position of the walker takes place at scale m. For large m, 
the space dependence is mild, since we will assume all through the paper the following 
smoothness property: 

Assumption (A.l). There exists a finite constant K such that \p(x,i,v) — p(y,i,v)\ < 
K\x-y\, x, y G R d , i G E, v G V. 

For technical reasons, which are explained in the paper, we impose the following ellip- 
ticity condition: 

Assumption (A. 2). For all x G M d , % G E and v G V, p(x,i,v) > 0. By continuity, 
M{p(x, i,v); x G [0, l} d , i G E, v G V} > 0. 

We also assume the environment to be ergodic and to obey the large deviations principle 
for Markov chains. We thus impose the following sufficient conditions: 

Assumption (A. 3). The matrix P is irreducible on E. Its unique invariant probability 
measure is denoted by \i. 

In particular, the following vector-valued function is smooth: 



the above expectations are taken over independent variables £, U, where £ has the distri- 
bution fi and U is uniformly distributed on [0,1]. 

For the deadlock time analysis, another assumption will be necessary (see (A. 4) in 
Section SD. 

2.2 Continuous Counterpart and Skorohod Problem 

Because of the reflection phenomenon, we briefly recall what the Skorohod problem is 
(we refer to [23] for a complete overview of the subject). For each continuous mapping 
w : t G [0, +oo) Kiu,G M. d , with w G [0, l] d , there exists a unique continuous mapping 
t G [0, +oo) i— ► (x t , kt) G [0, l] d x M. d , with k of bounded variation on any bounded sets, 
such that: 



W > , w t = x t + k t , k t = n s d\k\ s , \k\ t = / l {Xsed[0A]d} d\k\ s , (2.5) 



where n s G Af(x s ), M{x) denoting for x G d[0, l] d the set of unit outward normals to 
<9[0, l] d at x, that is 

N{x) = {!J£l d : \v\ = 1, v t = if x t G (0, 1), v t < if x e = 0, v e > if x £ = 1}. 

When x is in the relative interior of a face of the hypercube, J\f(x) is obviously empty. 

It can be proved (see again [23]) that, for every T > 0, the mapping \I/ : (w t )o<t<T | — ► 
(^t)o<t<T is continuous from Cr Oi i]d([0, T]; R d ) into itself with respect to the supremum 
norm (it is even l/2-H61der continuous on compact subsets of Cp^dQO, T}; M. d )); here 




d . 



(2.4) 
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and below, Ca([0, T]; M. d ) denotes the set of continuous functions from [0, T] to M. d with 
an initial datum in A. Moreover, if w is absolutely continuous, then x and k are also 
absolutely continuous (see [23], Theorem 2.2]). 

Equation (12.21) corresponds to a Euler scheme for a Reflected Differential Equation 
(RDE in short). An RDE is an ordinary differential equation, but driven by a push- 
ing process k as in (12.51) . For a given initial condition xq G [0, l] d and a given jointly 
measurable and x-Lipschitz continuous mapping b : M + x [0, l] d — > M. d , the RDE 



VT > 0, (x t ) <t< T = V 



Xq + J b(s,x s )ds 

/ 0<t<T. 



(2.6) 



admits a unique solution (see again [23]). This solution satisfies the equation 



Vi > 0, x t = x + / b(s, x s )ds — k 



»t 5 

with k as in (12. 5p . In this and /c are absolutely continuous. 

Reflected equations driven by Lipschitz continuous coefficients are stable. By 
Lemma 3.1], we can prove that for every T > 0, there exists a constant Ct > 0, such 
that, for any £o,2/o £ [0, l] d , the solutions (x t )o<t<T and (yt)o<t<T to ( 12.61) with xo and yo 
as initial conditions satisfy sup 0<i<T \x t — yt\ < Ct |^o — 2/o I - 

When b(s, x) = f(x), we denote by {xt°)t>o the unique solution to the averaged reflected 
differential equation 

Vt > 0, x t = x + [ f{x s )ds - k t . (2.7) 



3 Large Deviations Principle 



We now denote the process X by to indicate the dependence on the parameter m. In 



what follows, we investigate an interpolated version of the rescaled process (m 1 x[ m ^ 



namely 

X 4 (m) = (2n - Id) (m^xgSj + (t - m' 1 [mt^fim^X^ , ^^j , U Lmtl )) , t > 

= m-^g + ^-m-HmtJMm-^g^Lmtj^Lmtj), t>0 (3.1) 

We note that the hyperbolic scaling is different from the diffusive scaling in [8]. The 

(m) 



process (X| m ^) t >o is continuous and X^"^ = X^ m -* for any integer fcGN. 



3.1 Heuristics for the Non-reflected Walk 

We first look, for pedagogical reasons, at the non-reflected case. We thus consider 

Sl m) = m^S^ + (t -m' 1 [mt\)f(m' 1 Sy m t\^\ m t\,U\ nit {) , t > 0. 

(As for X, we indicate the dependence on m in S.) In light of Assumptions (A. 1—3), 
we expect the global effect of the environment process (£ n )n>o to reduce for large time 
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to a deterministic one. More precisely, if the initial position is such that Sq — > x as 
m — > oo, we expect (£' m ') m >i to converge in probability, uniformly on compact sets, to 
the solution x. of the (averaged) ordinary differential equation 

it = f{x t ) , x Q = x , (3.2) 

that is linim^oo p QjT (S^ m \ x.) = for all T > 0, where Po,t(0, = su P{|0t — ^tli £ ^ 
[0, T]} denotes the distance in supremum norm on the space C([0,T];IR d ) of continuous 
functions from [0, T] into M d . 

Loosely speaking, the Large Deviations Principle (LDP in short) for (S'. ( ' m ' ) ) m >i follows 
from the Freidlin and Wentzell theory [T6l Chapter 7] , or at least from a variant of it as 
explained below. The idea is the following. The irreducible Markov chain (£ n ) n >o with a 
finite state space obeys a LDP (see P, Theorem 3.1.2, Exercise 3.1.4]). In particular, the 
function H defined for x, a G M. d by 

1 " 

H{x,a) = lim -lnE l exp(a, I7 fc )) (3.3) 

k=l 



lim 



k=l 



exists and is independent of the starting point £o = i € E. Here, denotes expectation 
over C4) starting with ^ = h and the last equality is a direct integration on the i.i.d. 
sequence (i7 n ) n >i. From assumption (A.l) and finiteness of E, the limit is uniform in 
x , a on compact subsets of lR d and in i E E. 

In fact, H(x,a) is equal to the logarithm of the Perron-Frobenius eigenvalue (e.g., [SI 
Theorem 3.1.1, Exercise 3.1.4]) of the matrix 

Q(x, a) = [P(i, j)Ee^^] (y)gJ5xJJ . (3.4) 

Since the entries of the above matrix are regular and the leading eigenvalue is simple, H 
is continuous in x and infinitely differentiable in a. For x, v G M. d , the Legendre transform 
of H(x, ■) 

L(x, v) = sup{(a, v) - H(x, a); a G R d } (3.5) 

is no n- negative and convex in v. It is even strictly convex, in view of the differentiability 
of H(x, ■) (see [161 Chapter 5, (1.8)]). In particular for all x G M d , f(x) = V a H(x, 0) is 
the unique zero of L(x, •). Since \H(x, a)\ < \a\ for all x, a G M. d , we have (v G M d , \v\ > 
1) ^> L(x, v) = +oo. 

In some sense, the convergence in ( 13.31) corresponds to [T6l Chapter 7, Lemma 4.3]. By 
the regularity of H, we expect [T6l Chapter 7, Theorem 4.1] to hold in our framework. 
For x G M. d and for a sequence (x m ) m >i converging towards x, with mx m G Z d for all 
m > 1, we expect (5^"^) m >i to satisfy a LDP with m as normalizing coefficient and with 
the following action functional 

f T 

Iq,t{4>) = / L((j) s ,4> s )ds if 0o — x an d is absolutely continuous 
Jo 

= oo otherwise. 
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3.2 Large Deviations Principle for the Reflected Walk 

We now prove the LDP for the reflected walk. Generally speaking, it follows from the 
LDP for the process S. and from the contraction principle (see e.g. (9j Theorem 4.2.1, 
p. 126]). For this reason, we have first to make rigorous the previous paragraph. In 
what follows, we will see that the theory of Freidlin and Wentzell cannot be applied in a 
straight way. Indeed, for our own purpose (see the next section for the application to the 
deadlock time problem), we are seeking for uniform large deviations bounds with respect 
to the starting point. In [16], the authors obtain uniform bounds for systems driven by a 
Lipschitz continuous field /. Since our own / takes its values in a discrete set, it cannot 
be continuous. 

To overcome the lack of regularity of /, we follow the approach of Dupuis [UJ. The idea 
is to use a "uniform" version of the Gartner-Ellis theorem to obtain uniform bounds (see 
P, Theorem 2.3.6] for the original version of the Gartner- Ellis theorem). More precisely, 
we follow Section 5 in [TT]. In this framework, we emphasize that (X^)t>o is 1-Lipschitz 
continuous (in time) and adapted to the filtration (Gt = a (Ck,Uk, k < \tm\)) t >o. We 
consider the non-projected and projected versions 

Yt ] = Yl7l + {t-k/m)f{X§l^U k ) , 

z™ = n(z^l + (t- k/m)f{xi%, t k , u k) ) , 

with Yq = Z^ = Xq. They are also 1-Lipschitz continuous in time and adapted to 
(Gt )t>o- We let the reader check that, for all t > 0, \z[ m ^ — X^l^ < 1/m. Moreover, 
for t G [k/m, (k + l)/m), 

g( m ) _ r?( m ) I y ( m ) _ yM _ [fifM _ _ft-( m ) ] 
t k/m t k/m L t k/ml ' 

K (m) _ K (m) = , z {rn) y{m) _ y(mU _ ^{m) y(m) _ y (m)x 
t k/m \ k/m t k/m/ \ k/m t k/m) ' 

with Kq" 1 ^ = 0. Summing over k, we have zj. m ^ = y} 111 ^ — K[ m \ The process is of 
bounded variation on compact sets. If Z^ 1 ) G (0, l) d , — K ™) = for k/m < t < 
(k + l)/m. Otherwise, G d[0, l] d and - G M^(Z™) = ^(Z^). 

We deduce that Z^ is nothing but m(Y^) being the Skorohod mapping). Since 
Z^ and X (m) are close, it is sufficient to establish the LDP for y( m ) and to conclude by 
the contraction principle,. 

The LDP for y( m ) follows from [TTl Theorem 3.2] (up to a slight modification of the 
proof). Indeed, we can write 

y t M = y *S + (* " k/rn)f^(Y^) k/m + s^l^,U k ) , k/m < t < (k + l)/m , (3.6) 

with |£fc/^|oo < 1/m. This form is the analogue of the writing obtained in [TTJ p. 1532] 

for X e n . In (13.61) . we can choose an arbitrary initial condition y G [0, l] d for Y^ (it is 
not necessary to assume that my G Z d ). Similarly, we choose an arbitrary starting point 
i G E for £. To establish the LDP, we have to check Assumptions Al and A3 in [TT]. In 
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our framework, Al is clearly satisfied. We investigate A3. We first prove that H is x- 
Lipschitz continuous, uniformly in a (so that L is also x-Lipschitz continuous, uniformly 
in v). For x G M. d , a G W 1 and j G E, we set 

H (x,a;j) = \nQ2exp((a,v))p(x,j,v)). 

vev 

By (A.l) and (A. 2), H is x-Lipschitz continuous (uniformly in a and j). The Lipschitz 
constant is denoted by K' . By ( 13.31) . we obtain for x, y in M d and a G lR d 



x - y\. 



H(x, a) — (?/, a) 

n n 

lim -{lnE^exp(V# (x,a;a))] - lnE''[exp( V #„(?/, a; &))] } < 



fc=l fe=l 

It remains to estimate the conditional law of the increments of given the past. 

For a given t > 0, we consider a 1-Lipschitz continuous function <p G C([0, £]; 
with 0o G [0, l] d . From Subsection I2.2[ we know that \I/ is l/2-H61der continuous on 
compact subsets of Cr O) i]d([0, £]; M d ), so that we can find a constant 7 > such that 

p o ,t(^(^ (m) ),^(0)) < 7P^ 2 (F (m) ,0). For a G M d , S, A > and A G £ t m > with P(A) ^ 
and A C {p o ,t(^ (m) ,0) < 5} (so that A C {p 0it (*(y( m )), #(</>)) < 7<5 1/2 }), we have 

r[exp(m(a,yi m 2-F/ m) ))|A] 

r [(t+A)m\ -, 

exp( £ («,/(X^,a,^)))|A 



= e 2H E* 

,2[a| 



fe=|traj+l J 
L(t+A)mJ-l 

exp( ^ (a,/(A^,^,f4)))exp(# ^ 

fc=LimJ + l 

L(i+A)mJ-l 



< e 2|a| exp(K'A)E l 



exp( ^ (a, /(X, (m) , L/*)» exp(ifo(X 4 (m) , a; £ L (t +A)w j)) I A 

L fc=L<mJ+l 

By iterating the procedure, we obtain 
r[exp(m(«,F/ + m 2-F/ m) ))|A] 



< e 2|a| exp(A'A 2 m)E 

< e 2|Q| exp(A'A(A 2 m + 1))E 

< e 2|a| exp(A"'A(A + 7 <5 1/2 + 1 /m)m)E 



L(t+A)mJ 

exp( H {X { r\a-i k ))\A 

k=[tm\+l 

L(t+A)mJ 

exp( H (Z^,a;Q)\A 

k=[tm\ + l 

L(t+A)mJ 

exp( H (*(<f>) t ,a;£ k ))\A 



< e 4|Q| exp(A'A(A + 7 5 1/2 + l/m)m) supE^' 



k=\tm\+l 
|AmJ 



exp(£(a,/(*(0) t ,&,£4)>) 



fc=0 
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We deduce that, uniformly in the starting points y and i, uniformly in a on compact 
subsets and uniformly in (A, 0) satisfying A C {pq^(X i 0) < 6} 

limsupl/(mA)mP[exp(m(o;,y t S-F t (m) })|A] < H(V(<f>) t , a) + 2K'(A + 7 5 1/2 ). 

Similarly, we can prove a lower bound for the liminf. Even if written in a different manner 
(because of the Skorohod mapping \1/ and because of the conditioning - we give a precise 
sense to the right-hand side in jTT], A3, (3.6), (3.7)] - ), these two bounds correspond to 
those required in Assumption A3 in [H] (see the discussion on this point in (TTJ Section 
5]). 

We deduce that the sequence (y {m) ) m >i satisfies on C y ([0, T];R d ) (T > 0) a LDP with 
the normalizing factor m and with the action functional Iq T : >— > J Q T L(\l/(0) t , 4>t)dt if 
0o = y and is absolutely continuous and oo otherwise. We let the reader check that this 
action functional is lower semicontinuous on Cr 0)1 id([0, T]; ]R ) and that its level sets are 
compact for the supremum norm topology. By the "robust" version of the Gartner-Ellis 
proved in the LDP is uniform in y G [0, l] d . 

The uniformity of the LDP with respect to the initial condition is crucial. By the 
regularity of L in x (it is Lipschitz continuous, uniformly in a), it is plain to deduce 
that for any x G [0, l] d , for any closed subset F G C[ 0jl ]d([0, T]; R d ) and any open subset 
GeC md ([0,T};R d ) 



lim sup limsupm~ 1 lnP{Y (m),2/ G F\ < — inf Iq T {4>), 

S \° \y-x\<8 m^+oo <t>eF ' 

lim inf liminf m" 1 ln¥\Y {m) ' y G G\ > — inf IS T (d>), 

8\0 \y-x\<5 rn^+oo 1 J </>eG ' 



(3.7) 



where the notation Y^ ,v indicates that y( m ) starts from y (i.e. y ( j m - ) ' s/ = y). 

By the contraction principle (see e.g. Theorem 4.2.1, p. 126]), for any y G [0, l] d , 
(*(yW)) n ^ satisfies on C y ([0,T]; R d ) a LDP with m as normalizing factor and with the 
following action functional 

J o y0) = inf |2 L(*(^) a ,^)tZs, = 0} = inf |2 L{<j> s ^ s )ds, tfty) = 

(3.8) 

if 0o — y an d there is an absolutely continuous path ip such that = 0, and Jq T (0) = 
00 otherwise. 

Let us mention at this point that an alternative, more explicit expression of Jq t will 
be given below. Again, the action functional Jq T is lower semicontinuous on the set 
C[ 0) i]d([0, T]; [0, l] d ). The proof is rather standard and is left to the reader. We can also 
prove that the level sets J y , T (a) = {0 G C y ([0,T};R d ) : JJ T (0) < a}, for y G [0, l] d , are 
compact in the supremum norm topology. Moreover, (13.71) yields for any x G [0, l] d 

lim sup limsupm- 1 lnP{*(Y (m) ' 2/ ) G F} < - inf Jq T (0), 

5 \° |j/-a;|<(J m->+oo ' 

lim inf Hminfm _1 lnP{*(y (m) ' 3/ ) gG}> - inf 7^0), 
^X \y-x\<8 0eG 
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Now, we can come back to the sequence (X^) m >i. For a sequence (x m ) m >x of initial 
conditions in [0, l] d , with mx m G Z d and x m -> s, we have |X t (TO) ' Xm -*(y t (m),Xm )| 0O < 1/m 
for all £. By (13. 9p . we deduce 

Theorem 3.1 Assume that (A. 1—3) are in force and consider T > 0, x G [0, l] d and a 

sequence (x m ) m >i converging towards x, with mx m G [0,m] d D Z d for all m > 1. Then, 
the sequence (X^) satisfies on C([0, T]; [0, l] d ) a LDP iwito m as normalizing factor and 
Jq T as action functional. 

Following the proof of [91 Corollary 5.6.15], we deduce from (13.91) the following "robust" 
version (the word "robust" indicates that the bounds are uniform with respect to the 
initial condition) 

Proposition 3.2 Assume that (A. 1—3) are in force and consider T > and K a com- 
pact subset of [0, l] d . Then, for any closed subset F of C([0, T], [0, l] d ) and any open 
subset G o/C([0,T], [0,l] d ) ; 

limsup[m -1 ln sup F x {X {m) G F}~\ < — inf inf J% T (4>), 

m-*+oo xeK,mxeZ d ' xeK 

liminf \rri- 1 In inf F x {X {m) G G}] > — sup inf J% T (<j>). 

m— >+oo x£K,mx€Z d x ^k <t>£ G 

3.3 Law of Large Numbers for the Reflected Walk 

We discuss now the zeros of the action functional. We first consider the solution (x°t°)t>o, 
x G [0, l} d , to (12771) . Setting 

Vt > 0, y t = x + [ f( X x s °)ds, 
Jo 

we have, for T > 0, (xt°)o<t<T — ^((yt)o<t<r)- Since the path i G M + i— > y t is absolutely 
continuous, we deduce 



</ W°)< [ T L(y s J(y s ))ds = 0, 
Jo 



so that x x ° is a zer ° of Lq^. In fact, this is the only possible zero for the given initial 
condition xq. Consider indeed another path with values in [0, l] d , such that Jq ° t (4>) = 0. 
The set of absolutely continuous functions ip such that ipo = Xo, 

T 

L((p s ,^ s )ds < 1 and #(-0) = 0, 

is compact. Since the functional ip t— > J Q T L((j) s ,ip s )ds is lower semicontinuous, it attains 
its infimum on this compact set. Hence, there exists an absolutely continuous function ip 
such that -00 = Xo and 



[ L((f) S} ip s )ds = and *(V>) 
Jo 
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It is clear that ip t = f{4>t)- Since \I/(-0) = (fi, there exists a process k as in (12.71) such that 

Vt G [0, T], <fi t = x + [ K<p s )ds - k t . 

Jo 

This proves that = x x ° up to time T. 
A direct consequence is the following 

Corollary 3.3 Assume that (A. 1—3) are in force and consider a sequence (x m ) m >i in 
[0, l] d , with mx m G Z d for all m > 1, such that x m — > x as m — > +oo. Then, the sequence 
of random paths (X^) m > 1; with = x for all m > 1, converges, in probability, 

uniformly on compact time intervals to the solution (Xt)t>o °f the (averaged) reflected 
differential equation ( 12.71) . with \o = x - 



3.4 A Different Expression for the Action Functional 

Following [10] , we write the action functional J^t in a different way. We recall that M{x) 
denotes the set of unit outward normals to d[0, l] d at a point x on the boundary. We 
define the function L ref by L ref (x, •) = L(x, ■) for x G (0, l) d , and for x G <9[0, l] d , 

+oo if 3n G Af{x) : (v, n) > 

L Icf (x,v) = \ L{x,v) if Vn G JV(a;) : (v, n) < ( 3 10 ) 

inf L(x,v + f3n) otherwise 

/9>0,neAT(2:),n±j; 

The last case occurs when (v, n) < G M{x) and 3n' G M{x) : (f , ra') = 0. Then, the 
motion takes place on the boundary, in the sense that, for e > small enough, x + ev 
remains in the face orthogonal to n'. Observe that, in contrast to L(x, ■), the function 
L ref (a;, ■) may be non convex and discontinuous for x G d[0, l] d . 

Theorem 3.4 Assume that (A. 1—3) are in force. If is absolutely continuous it holds 

T<j>0 I i\ / rref 



J^)= / U°\(j> tl <t> t )dL 



o 

If <p is not absolutely continuous, then Jq° t {4>) = oo. 

□ By Theorem 2.2 in [23J, we know that ^(ip) is absolutely continuous if ip is abso- 
lutely continuous. In particular, if is not absolutely continuous, there cannot exist an 
absolutely continuous ip such that ^(ip) = <fi. 

Assume now that <fi is absolutely continuous. Then, there exists at least one absolutely 
continuous path ip such that ^(ip) = 4>, namely itself with k = 0. We thus denote by 
ip an absolutely continuous path such that = ^(VO and set k = ip — 0. Then k is also 
absolutely continuous and k t = (3 t n t with f3 t = d\k\ t /dt > (= if 4>t ^ d[0, l] d ) and 
n t G Af(<f) t ) if 4>t G d[0, l] d . Moreover, for a.e. t, for all I G {1, . . . , d}, (0t)^l{(<^e{o,i}} = 
so that (fit _L kt- Hence 

L{<t> t ,^ t )dt> / L ref (0;,0 t )cft. 
o Jo 
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This proves that 

JoM > [ L* ei (<j> t ,j> t )dt. 

Jo 

We investigate the converse inequality. If the right-hand side is infinite, the proof is over. 
Thus, we can assume that it is finite, in particular L Tet ((f) t ,<p t ) < oo for almost every 
t G [0, T\. It is enough to construct some ip with ^(ip) = (ft and L(<f> t ,tp t ) = L ref (0t,0t) 
a.e.. For times t's when <p t G d[0, l] d and L vef ((p t , 4>t) < oo is given by the last line of 
(I3.10p . the infimum is achieved at some pair (3 t > 0, n t G Af(4>t) (this pair is unique by 
the strict convexity of L). Since H(x,a) is bounded by \a\, \v\ > 1 L(x, v) = +oo. 
We deduce that \<p t + f3 t n t \ < 1, so that \(3 t \ < 1 + \<pt\- For other times t, set (3 t = 0,n t 
arbitrary. The mapping t G [0, T] i— > f3 t is clearly measurable and integrable. Hence, we 
can define ip t = 4>t + fit^t, V'o — 0o an d 4>t = V'o + Jo^Psds- The function ^ meets all our 
requirements. ■ 



4 Analysis of the Deadlock Phenomenon 

We now investigate the deadlock time of the algorithm. Fixing a real number I G (0, d), 
we define 

G = {x G [0, l} d : |x|i < £} , and dG = {x <E [0, l] d : \x\\ = £} (4.11) 

its boundary relative to [0, l] d . We also define the discrete counterparts at scale m, 
G (m) = | x e ( m -i^ rf ) n [0,1]^ : |x|i < m- 1 ^}, G^ = {x G (m"^ ') n [0, l] d : |x|i < 
m- l \mi\} and 9G( m ) = G (m) \ G^ = {x G (m _1 Z d ) n [0, l] d : \x\i = m -1 ^]}. The 
deadlock time for the process is 

r ( m ) = — inf {n > : |X n |i = [rrd\ } = inf {t > : X t (m) G <9G (m) }. 

We consider the following simple situation: 

Assumption (A. 4). The point is the unique equilibrium point of the RDE (12.71) . It 

is stable and attracts the closure G = G U dG, that is, for all xq G G and t > 0, Xt £ C 
and lim^oo = 0. 

Example (I5.58P given below satisfies the previous assumption provided that gx,92 are 
(strictly) positive on (0, 1]. 

Quasi-potential. The function 

V(x,y) = inf{J^ T (0); O = x, <p T = y, T > 0} 

is called the quasi-potential. It describes the cost for the random path starting 
from x to reach the point y G G at some time scaling with m as m becomes large. 
(We emphasize that, here and below, the notation Jq T (<P) implicitly assumes that is a 
function from [0,T] to [0, l] d .) 
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Proposition 4.1 Under Assumptions (A. 1—3), there exists a constant C > 0, such that, 
for all x,y G [0, l] d , with A = \x — y\\ > 0, the function ip : t G [0, A] t— > x + £(?/ — x)/\ 
satisfies ipo = x, ip\ = V and </o,a(V0 < CX. In particular, V(x,y) < C\x — y\\. 

□ Proof. By (A. 2) and for all x G [0, l] d and a G M d , H(x,a) > lnfcexpflaloo)) = 

ln(c) + |a|oo, with c = inf|jo(z, i,v); z G [0, l] d , z G u G V} > 0. Hence, for all v G R d , 
L(x,v) < swp a {(a,v) - \a\oc} -ln(c) < sup Q {|a|oo(|t;|i - 1)} -ln(c) < -ln(c) if |u|i < 1. 
The proof is easily completed. ■ 



4.1 Deadlock Time and Exit Points 

We define the minimum value of the quasi-potential V(0, •) on the boundary of G by 

V = inf{J o ° T (0); O = 0, |<Hi = t, T > 0} 
and the set of minimizers 

M = {y G dG : V(0, y) = V}. (4.12) 
By Proposition 14.11 V is finite. A consequence of Theorem 13.11 and Proposition 14.11 is 

Theorem 4.2 Assume that (A. 1—4) are in force and consider a sequence (x m ) m >i i n 
G, with mx m G 7L d for all m > 1, such that x m — > x G G. Then, 

E x Jr (m) ] = exp[m(V + o(l))] (4.13) 

as m — > oo. Moreover, for all positive 5, 

lim F Xm { exp[m(V - 5)} < r (m) < exp[m(V + 5)}} = 1. (4.14) 

Finally, for all e > 0, it holds 

F Xm {d(X i ™2 ) ,M)<s}^l asm^oo, (4.15) 

where d(X^ )} AA) denotes the distance from X^^ to the set M.. 

□ Proof. The proof follows the standard theory of Markov perturbations of dynamical 
systems in [T6l Chapter 6]. For the sake of completeness, we provide the main steps 
according to the very detailed scheme in P, Section 5.7] (Section 5.7 is devoted to large 
deviations for stochastic differential equations with a small noise). 

We define, for x G G, V(x,dG) = inf{Jo T (0); O = x, \4>t\i = £, T > 0}, so that 
V(0, dG) = V. We also define the ball in the lattice orthant of mesh 1/m, 

£M,+ = { zE ( m ~ l Z d ) n [0, l} d : \z\!< m- 1 [mp\ }. 
In the whole proof, we assume that < 2p < £. 
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Lemma 4.3 For any r] > and for any p > small enough, there exists T < +oo, such 
that 



lim inf m -1 In inf F x {r {m) < T } > -V 



r/. 



□ Proof. We first fix a small t] > 0. By the definition of V, we can find So > and 
0° G C([0,S ], [0,l] d ), with 0° = 0, such that J^ So ((j) ) < V + r] and So G <9G. By 
Proposition 14.11 and by the additive form of J, see Theorem 13.41 we can extend after 
S to leave G at low cost, and assume that 0([O, So]) H <9G 7^ and 8 = d(</>s , G) > 0. 

For x G [0, l} d , \x\i < 2p, we can find by Proposition O a path £ G C([0, 2p\; [0, l] d ) 
such that Co = x, (,2 P = and Jo 20(C) < Cp- By concatenating £ and 0, we obtain a path 
X . For p <T)/C, it satisfies 

Jo,T o (0 a; )<^ + 2r/, 

with T = S + 2p. Now, the set 

*= (J {^GC([0,T ];[0,l] d ): p T (V>, 4> x ) < 8/2}, 

xe[0,l] d , \x\i<2p 

is an open subset of C([0,T o ]; [0, By Proposition E21 

liminfm" 1 inf lnP a! {r (m) < T } > liminf m" 1 inf lnP x {X (m) G ^} 

> - sup inf Jo }To (ip) 

xG[0,l] d , Hi<p^ 6 * 

>- sup J- To (0-)>-y-2r / . 

xG[0,l] d , \x\i<p 

This completes the proof. ■ 

Lemma 4.4 Lei a p = inf{t > : X t (m) G i?J m) ' + U dG^}. Then, 

lim limsup[m _1 hi sup ¥ x {a p > £}] =—00. 

*^+°° m^+oo xGG( m ' 

□ Proof. For x G B p + , there is nothing to prove. Now, as in the proof of P, Lemma 
5.7.19], we can define for t > the closed set * t = {0 G C([0,t]; [0, l] d ) : ip s E G \ 
B+ 2 , Vs G [0,t]}, where S+ is the ball in the orthant, B+ /2 = {z G [0, l] d : |z|i < p/2}. 

For Xq" 1 ^ G G^ m ^ and m large, a p > t implies (xi"^)o<s<t G ^ By Proposition 13.21 
lim sup [mT 1 In sup ^xWp > 0] 

m-»+oo xGG('")\ J B< m) ' + 

< lim sup [m" 1 In sup P x .{X (m) G * t }l < - inf inf J %(^) = - inf JS(V0- 



P/2 



Using the stability of the solutions to (12.71) (see Subsection 12. 2p and the additivity of the 
action functional (see Theorem 13 .4p . we can complete as in [9]. ■ 
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Lemma 4.5 Let N be a closed subset, included in dG. Then, for every e > 0, 
limlimsup[m -1 ln sup P J/ {dist(X( m) , N) < e}] < - inf V(0,z) + 5 e , 

m-t+oo _c,("i).+ P zeN 

V fc,5 2p 

wi/i lim £ _>o^e — 0. Here, Sp m ^' + = {z G m -1 Z d D [0, l] d : |z|i = m~ l \_pm\} is the sphere 
in the lattice orthant with mesh 1/m. 

□ Proof. The proof is the same as in P, Lemma 5.7.21], except the application of 
Corollary 5.6.15. For T > 0, we can define, as in [3], $ = {(p G C([0,T]; [0, l] d ) : 3t G 

[0,T], 4 G JV}. If u p < T and dist(X#°,iV) < e, then p ,T(^ (m) , $) < e. So that, 
Proposition 13.21 yields 

limsup [m~ l In sup ¥ y {a p < T, dist(X^ } , N) < e}] 

< - inf inf J o y T (0) = - inf { J&(0); d(0 o , Sj) < e, ^(0, $ ) < e}, 

d(y,S+ p )<e d(4>,<3>)<e ' 

with 5+ = {2; G [0, l] d : |z|i = p} is the sphere in the lattice orthant. Using the 
semi continuity of J, the reader can check that (see [HI Lemma 4.1.6]) 



lim inf inf JfrW = inf inf Jf T (0). 
The end of the proof is the same. ■ 

Lemma 4.6 Lei K be a compact subset of [0, l] d included in G^ for m large. Then, 

hm inf F Xm {X^ G 5W> + } = 1. 

□ Proof. The proof is the same as in [HI Lemma 5.7.22], up to the infimum over the 
compact set K. By (A. 4) and by the regularity of the flow (t,x) G M+ x [0, l] d 1— > 
Xf, the hitting time T = inf{t > : Vx G [0, l] d , |xf|i — p/2} is finite. Moreover, 
in£te[o,T\,xeK dixti ®G) > 0- Using Corollary 13.31 it is plain to conclude. ■ 

Finally, we have the following obvious result 
Lemma 4.7 sup F x { sup |X 4 (m) - x\ > 2p} = 0. 

x .g G (m) o<t< P 

It now remains to follow the proof of [HI Theorem 5.7.11]. The crucial point to note 
is the following: and a p take their values in m _1 N and are stopping times for the 
filtration (^fmi|)*>°' I n particular, the Markov property (for (£, X)) applies quite easily. 
For example, for x G G^ and s,t in N* (and thus in m _1 N*), 

P»K < t, X™ G 5™'+, <t + s} 

> V x {o p < t, X™ G 5^'+} inf P;{r(™) < s}, 
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so that 

P*{r (m) < t + s} 

> ¥ x {a p < t, G 5< m >'+ r< m > < t + s} + F x {a p < t, X™ G 3G^} 

> p.K < *} inf F U r(m) ^ s >- 

This shows that (5.7.24) in [9j holds. Similarly, for t G N* and fceN, 

P^{r (m) > (Jfe + l)t} < P x .{r (m) > kt) sup P^{r (m) > t}. 

yGG (m >,i£E 

Now, the upper bounds in ( 14. 13ft and ( 14.141) can be derived as in [9]. 

Turn to the lower bounds. Following [9J, we introduce the following notations (pay 
attention to that m in [9] refers to a complete different parameter than in our case): 

#o = 0, r n = m£{t > 6 n : X 4 (m) G flM,+ y = inf{f > r n : X t (m) G S$>' + }, 

(4.16) 




Figure 1: The path up to the deadlock time (d = 2,£= 1.7). Spheres S p , S 2p 
are indicated by dashed lines. The seven large dots on the path are the locations at times 
@o = t~o = 0, 6>i, ti, 9 2 , t 2 , 6*3, r 3 = r^. The last part of the curve is the terminal segment 
Terseg defined in the proof of Theorem 14.81 

with 8 n+ i = +oo if X^ G dG^ m \ These stopping times are indicated in Figured) It is 
plain to obtain (5.7.26) of [§] (with the Markov property and Lemma [4.51 with N = dG 
and e as small as necessary) as well as (5.7.27) (with Lemma 14.71) . The end of the proof 
of the lower bound just follows the strategy in [5]. 

Turn to the second statement in Theorem 14.21 This is a particular case of b) in [9] . Set 
N = dG H {x G [0, l] d : dist(x, M) > s}. It is a closed set. Then, for s' > 0, we can 
focus on 

sup F y {dist{Xl m) ,N) <e'}. 
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Setting Vn = inf y€ 7v V(0, y), we deduce from Lemma l4~5l that for p, e' > small enough 
and for m large enough 

sup P,{dist(X(;\ N) < e'} < exp[-m(V N - rj)}, 
with 7] < (Vn — V)/3 < 0. Then, we can follow the proof in [9] and prove that for 

lim P 1 . m {dist(X^ m ) ) ,iV)<e'} = 0. 

m— >+oo ' 

Since dist(X^ } , 9G) < C/m, we complete the proof. ■ 



4.2 Generic Behavior Leading to Deadlock 

From (14.151) we observe that when M. reduces to a single point y*, the location of the 
process X( m ) when exiting G converges to y*. We can extend this observation from the 
exit point to the path itself before it exits G. To do so, we first need to extend the action 
functional to any interval of R, which can be done in a trivial way thanks to Theorem 13. 4t 
for any continuous path (ipt)t<o, with lim_ 00 -?/> = 0, we denote by J_oo,o(VO the integral 
of U (ipt,ipt) from — oo to 0. Since is a fixed point for the limit RDE by Assumption 
(A.4), we have L ref (0,0) = 0, and then 

inf{J° T (0); O = 0, <fi T = y, T > 0} = inf{J_ oo0 (V»); lim^ = 0, ip = y}, 

— oo 

for y G G. Indeed, for all T, as in the left-hand side, the path ip given by ip t = 4>t+T 
for t G [— T, 0] and ip t — for t < —T is such that J-. O o,o('0) = Jq,t{4>)- This proves 
that the left-hand side is greater than the right-hand side. Conversely, for a path ip with 
lim.oo ip = and ip = y, we can find, for every 5 > 0, T < such that \iPt\ < By 
Proposition 14.1} we can find a path 6 from [0, 5] into [0, l] d , with 9 = and 9$ = ipx, 
such that Jq s(0) <C5. Concatenating this path to the restriction of the path -0 to [T, 0] 
(up to a trivial change of time in if>), we obtain a new path 0. It is defined on [0, T + 5] 
and satisfies O — 0, 4>t+s = V and Jq T (4>) < C<5 + J- o,o(V ; )- This proves that the two 
infimums are equal. 
Now, we can state the convergence result of the exit path. 

Theorem 4.8 Under Assumptions (A. 1—4), assume uniqueness of the optimal path to 
exit G from 0, i.e., assume that M. = {y*} and that there is a unique if : (— oo,0] — > G, 
<f((— oo,0)) C G, minimizing J_ DOj0 (<^) subject to ip = y*,\im t ^_ 00 ip t = (in such a 
case, ip is also the unique minimizing path with values in [0, l] d - and not only in G - ). 
Let K be a compact set, included in G, and containing a neighborhood of the origin. We 
denote by a^f 1 the last exit time before of from K n (m -1 Z d ). Then, for any 
sequence (x m ) m >i, x m G G^ and x m — > x G G, and any e > 

hm F Xm {3t G [4V TO >], |X, (m) - ^_ r(m) | > e} = 0. 
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□ Proof. Our proof is inspired by [H Section 2, Chapter 4]. We keep the notations 
introduced in the proof of Theorem 14.21 In addition, we define v = max{n > 1 : 9 n < 
T (m)y jf T (m) _ r ^ we ge ^. v _ g ^y e denote D y Terseg (X ^ m > ) the terminal "segment" of 
the path X^ m \ that is, the restriction of to the interval [9 U , but shifted 

in time to the interval [0, t v — 9 U }. More precisely, if we denote by Q t the shift operator, 
i.e. GiVK 5 ) = ^(s + then Terseg(X^ m ^) is defined as the restriction of Qg v (X^) to 
[0,r u -9 u ]. 

Fix £ > 0. For y G Bf^ ,+ and L G N*, we have r = and 
PjPo,r,-^(Terseg(X( m )),e e _ T ^) > e} 

L 

< P y {r< m > > r L } + X)P«{r (m) = r fc ,p efc , Tfc (X( m ), 9_ Tfc ^) > e} 



fe=i 

L 



(4.17) 



<P,{r^ >r L } + ^P y {X^ GaG( m ),p, fc , Tfc (X( m ),e_ T ^) >5}. 
fc=i 

Focus on the second term. The Markov property yields 

L 

^ y {Xif G dG^\pe k , Tk (X^\Q. TkV ) > e} 



k=l 



< L sup Pi{XM G ^.^(XW.e^) > e}. 

For T > 0, we can bound the last quantity as follows 

F Z {X^ G dG^\po,, p (X^\Q^ pV ) > e} 

< K{°p ^ T}+F Z {X^ G dG^\a p < T,p 0lCT ,(* (w) ,©-^) > e}. 



(4.18) 



(4.19) 



Now set, for T,r > 0, T T (r) = {i> G C([0,T]; [0, l] d ) : ^([0,T])n<9G ^ 0, p_ T ,o(©rV', ¥>) > 
r}. We then recall the following result in [3] (see Lemma 2.8, p. 105, the proof relies on 
the uniqueness of <p and is exactly the same in our setting, except (4), p. 106, which has 
to be read liminf fe ^ +00 d_ Tkfl (Q Tk g k , <p) > 0): 

Vr > 0, 3a > 0, VT > 0, inf J r(V0 > V" + a. 

i>er T (r),ip =o 

We now consider T, r > and ■?/> G Ty(r) with |^q| < 2p. We then prove that the 
above lower bound still holds for p small enough. Indeed, we can consider a path if>, 
with ipQ = 0, ips = i>o and ipt+s — 4>t for t G [0, T]. Using Proposition 14.11 we can 
assume that S < Cp and that J 0j t+s(4>) < Cp + J 0) r(^)- We choose Cp < a/2. Since 
Po,t+s(4>, Q-(T+S)(p) > r, we have J QjT+s (4>) > V + a. Finally, Jo,t(V0 >V + a/2. 

We now choose r = e/2. For the corresponding a > 0, we choose Cp < a/2 as above. 
Then, by means of Lemma 14.41 we can pick T large enough so that for m large enough 

sup F z {a p > T) < exp(-m(V + 1)). (4.20) 



z&S. 



2p 
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Now, for < s' < e/2, 

sup V,{XM G dG^ m \a p < T,p ^(X^,Q^ pV ) > e} 



zes. 



(m),+ 
2p 



< sup F x {p 0tT (X^\A T (e/2,2p))<e'}, 



zes. 



(m),+ 
2p 



where A T (e/2,2p) stands for the set of continuous functions from [0,T] into [0, l] d , with 
\4>o\ < 2p, for which we can find t G [0, T] such that the restriction of ip to [0,t] belongs 
to T t (e/2). This is a closed set. Hence, Proposition 13.21 yields for e' small enough and m 
large enough 

sup P,{lH G 3G< m >, a p < T, p ,. P (^ m \ 9^) > 5} 
< exp[-m( inf inf Jn T (</>) - a/12)] 

V zeS + d(«^,A T ( £ /2,2p))< e ' Ul ' V 

<exp[-m( inf J n 0o T (0) - a/12)] < exp[-m( inf J^(0) - a/6)], 

d(0,A T (e/2,2p))<e' U,J </>eA T (e/2,2p) U,i 

(4.21) 

the last inequality following from j9j Lemma 4.1.6]. For all (f) G At(e/2, 2p), there exists 
t G [0, T] such that the restriction of to [0, t) belongs to Y t {e/2). We deduce that 
4° T {<I>) > 4°M >y + a/2. Finally, by (EE}, (EE]), gl9]), g2Q|) and g2H), 

n{Po,r,-^(Terseg(X( m )), Qe^f) > e} < ¥ y {r^ > r L } + 2Lexp(-m(V + a/3)). 

We can conclude as in the proof of [9j Theorem 5.7.11, (b)]. We can find a constant C 
such that 

sup P,{p ^-aTerseg(X( m )), Q 0v -t v <p) > 4 
yeB p m) ' + 

< CL- 1 exp(m(V + a/6)) + 2L exp(-m(F + a/3)). 

We then choose L = [exp(m(V + a/4))J. For an arbitrary initial condition in G, we 
conclude as in the proof of [HI Theorem 5.7.11, (b)] by means of Lemma 14.61 (and the 
Markov property). ■ 



4.3 Exponential Limit Law for Deadlock Time 

Since the exponential law is the generic distribution for rare events, it appears naturally 
in the following refinement of Theorem 14.21 (see e.g. [301 Theorem 5.21]). 

Theorem 4.9 In addition to (A. 1—4), assume that the matrix P 2 is irreducible and that 
there exists a constant n > such that for all x,y G [0, l] 2 and i G E 

^2 \p( x ' i i u ) ~P(y^h u )\ + ^2(p( x ,h u ) ~ P{v, h «))sgn((x -y,u)) < -K\x-y\i 

u£A,U-Lx—y uGA 

(4.22) 
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(As usual, sgn(-) denotes the sign function, with sgn(it) = u/\u\ foru 7^ and sgn(O) = 
0.). Define T^ = min{£ > : Pj,(r (m) > t) < e -1 } for i G E and m > 1. Then, for any 
sequence of starting points (x m ) m >i in G, with x m — > x G G as m — > +oo, 

t/ie /aw of r 1 ^ jT % m under ¥ % Xm weakly converges to an exponential law of mean 1. 

In what follows, we will prove that, for any i,j G E 1 , T^/T^ — > 1 as m tends to +oo. 
In particular, the image law (r^ m ' /T^ l )(¥ i x ) weakly converges to an exponential law of 
mean 1 for any i,j G E. 

Condition (14.221) is not empty : Example (I5.58P given below fulfills (14.221) if g±,g2 are 
strictly increasing with g[,g' 2 > k' a.e. for some k' > 0. 

□ Proof. The following result is the analogue of [501 Lemma 5.22]. Its proof is deferred 
to Section 17.11 

Lemma 4.10 There exists 5 > 0, such that, for alii G E and S > 0, 

lim sup sup |PL /m {r (m) > tm 2 } - V y/m {r^ > tm 2 }\ = 0. 

m ^+°° \x\ 1 ,\y\ 1 <Sm,\x-y\ 1 e2N t>S 

With this lemma at hand, we can prove 

Lemma 4.11 For alii] > and S > 0, we can find a sequence (e m ) m >i of positive reals, 
tending to as m —>■ +oo ; such that for all i,j G E, 

Vi > S, P^{r (m) > tm 2 } < P J {r (m) > tm 2 - r]m} + e m 

□ Proof of Lemma HTU For i G E, we set #j = inf{/c G 2N : X k = i}. Since P 2 is 
assumed to be irreducible, it is a finite stopping time. For 5 as in Lemma \A. 101 and rj < 5, 

P^{r (m) > tm 2 } < P^,{r (m) > tm 2 ,^- < r/m} + P l {^- > r]m} 

< sup P^,{r (m) > tm 2 - r]m} + P l Q {^ > r]m}. 

{|z|i<<5m,|a;|ie2N} 

It is clear that lim m _» +00 Pq{^ > ^m} = 0. By Lemma f4. 101 the proof is easily completed. 
■ 

We now complete the proof of Theorem 14.91 We keep the notations introduced in the 
proof of Theorem 14.21 Following [3D1 Lemma 5.23], we can set for i EE 

Vt > 0, F^' l {t) = P*{r (w ) > tT m } = P*{r (w ) > m' 1 [mtT l m \} . 

By Theorem I4.2[ for every 5 > 0, we have lim m ^ +00 T % m exp[— m(V — 5)} = +oo and 
lim m ^ +00 T^ exp[— m(V + 5)] = 0. Moreover, by the Markov property, for j G E, p < £ 
and t > 0, 

sup P Ira {rH > m -i \_mtT m \ , a p < m' 1 [2jJ } 

< sup K m {a p <m-\T m \} sup F k y {r^ > m -\[mtT m \ - [TJ)} 

< sup P[{rW > r^flm^ - ^j)}. 
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In the above supremum, we aim at applying Lemma HUU] to the starting points and y (p 
being small enough). There is no difficulty if \y\\ G 2m _1 N. If \y\± G (2m _1 N + m _1 ), the 
Markov property yields ¥ k y {r^ > m" 1 ([mtT^\ - [7* J)} < sup ]z _ yh=1/Tn>k , eE Pf {r^ > 
m^dmtT^l — \Tln\ — 1)}, so that we can still apply Lemma \A 101 By Lemma l4~il 
we deduce that we can choose p small enough and find some sequence (S m ) m >i with 
lim m ^ +00 5 m = such that 

sup KJr {m) > m- 1 [mtT m \ } < sup P fc {r^ > m" 1 {\mtT m \ - [V m \ - 1)} + 5 m 

< F {r< m > > m-\[mtT m \ - 2^J)} + S m , 

(4.23) 

the second line following from Lemma [4.111 The Markov property yields for t, s > 0, 

> m -\m{t + s)T m \) 
<V {r^ >m-\[m{t + s)T m \ - [mtT m \)} sup HJt™ > m' 1 [mtT m \ } 

< W {t^ > m-\\m{t + s)Ti\ - [mtT m \ )}P*{r^ > m-\[mtT m \ - 2[T m \)} 
+ 8 m . 

(4.24) 

We can prove the converse inequality in a similar way. For any compact subset K C G, 
we deduce from Lemmas 14.41 14.61 14.101 and 14.111 that, for tm > 1, (up to a modification 
from line to line of the sequence (<5 m ) m >i - which may depend on K - ) 

inf n{T^>m~\mtT m \} 



> inf Pi> M > ™~ l \pOr m \ , °p < \Tl\ , e ^ m) ' + ) 
x m exnGH m 

> inf Pi>, < m- 1 LT^j , Xf) G Z^> + } 



(4.25) 



x inf P£{r (m) > m _1 Lmt2^J} 

> (1 - U [Po{^ (m) > m-^Lm^iJ + [T^J)} - 5 m ] 

> V {r^ > m-\[mtT m \ + [T m \)} - 5 m . 

Now, for t, s > 0, fH^oD yields 
Par( m )>m- 1 Lm(t + S )T^j} 

inf PL^^m-HmtTj} (4 . 26 ) 

> F {r<™> > m-HW + S )7^J - L^Tj),X^ 1(Lm(t+s)T4J _ Lm ^ J) G K} 
x P^{r( m ) > m~\[mtT m \ + LTj)} - 5 m . 

By (A. 4), for any starting point x G G, Xt° G G for t > 0. In particular, d(xf°, dG) > 0. 
By the stability property for RDEs driven by Lipschitz continuous coefficients, we have 
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mi xo£ Q d(xi° , dG) > 0. In other words, we can find a compact subset K$ C G such that 
Xi° G i^o for any x G G. We denote by e = d(K , dG) > the distance from K Q to dG. 
By Corollary 13.31 

lim sup F J X {d{x[ m \dG) < e/2} = 0. (4.27) 



By the Markov property 

T m-i(Lm(«+s)^J-MT4J)' 



P* {t<™> >m- 1 (Lm(t + S )Tj - [m*CJ ), d(X ( ™X (lm(t+s)T < ,_. iT , |V SG) >e/2} 



-F {r^ >m- 1 (Lm(t + s)Tj - [mtT^ ^d^^^j.^j.SG) < e/2} 
>P J {r( m ) >m- 1 (Lm(t + s)T m J - LmtTj)} - sup P^MX^, 9G) < e/2}. 

We can plug # = {2: G G : 9G) > e/2} in flQgj) . By ffl~2Tj) and the above inequality, 

P* {r< m > > rn-HmCt + s)Tj} > P {rW > m'^Hi + s)Tj - [mtT m \)} 
x P {r^ > m-\[mtT m \ + [T^J )} - 5 m . 

By (Ol and (jOgjl . 

F (m ^(s + 5 m )F^ l (t + S m ) -5 m < F^\t + s)< F^\t - 5 m )F^\s - S m ) + S m , 

so that limsup m ^ +oc F^ m ^ l (k + e) < e~ k for k G N and e > 0. In particular, the sequence 
r ( m )/j^ i s tight. Up to a subsequence, it converges in law. The limit distribution function 
is denoted by F. Up to a countable subset of (0, +oo), F^ n > , ' l {t) converges to F(t). Hence, 
we can pass to the limit in the above inequality. For all rj > 0, 

F{t + r])F{s +77) < F(t + s) < F(t - r])F{s - 77). 

It is plain to deduce that the limit distribution is the exponential law with mean one. 
By (14.231) and (|4.25p . this is true for any starting point. Moreover, for all j G E, 
(r^/T^) (Pq) weakly converges to the exponential law with mean one. Since (r^/^m)(^o) 
weakly converges to the same distribution, we deduce that T^/T^ — > 1 as m — > +00. ■ 



4.4 Hamilton-Jacobi Equation for the Quasi-Potential 

In practice, it is important to compute the quasi-potential 1^(0, x) as well as the optimal 
paths. (In what follows, we write, for the sake of simplicity, V(x) = V(0,x).) 

In [THl Chapter 5, Theorem 4.3] and [HI Exercise 5.7.36], it is shown that the quasi- 
potential is characterized through a Hamilton-Jacobi equation of the form 

H(x,VV(x)) = 0. 

Loosely speaking, the equation for the quasi-potential has the same structure in our 
setting. However, due to the reflection phenomenon, it satisfies some specific boundary 
condition. 
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Form of the Equation. Here, we specify both the equation and the boundary con- 
dition in the viscosity sense, the notion of viscosity solutions being, in a general way, 
particularly well adapted to optimal control problems. (See for example [5] or [6] for a 
review on this connection.) Indeed, the quasi-potential is nothing but the value function 
of some optimal control problem. In the formula (13.81) . L(<p s ,ip s ) may be interpreted as 
some instantaneous cost at time s when the trajectory <ft is driven by the control ip. The 
controlled dynamical system obeys the rule: Vi > 0, fa — ipf — kt, with k as in (12. 5ft . 

Proposition 4.12 We assume that (A. 1—3) are in force. Then, for every x G (0,1)^ 
and every continuously differentiable function 8 on a neighborhood U C (0, l) d of x, 

H(x, V9(x)) < if V — 9 has a local maximum at x, 

I \ (4-29) 

H[x, S79(x)) = if V — 9 has a local minimum at x. 

Moreover, for every x G d[0, l] d and every continuously differentiable function 9 on U n 
[0, l] d , U being a neighborhood of x, 

H(x V9(x)) > if { Wn G M{ - X) i & e ( x \ n ) ^ °' 

^ ' ' \ V — 9 has a local minimum at x on U D [0, l] d , 

, (4-30) 

H(x VB(x)) < if { Vn G N{X) ' n) ~ °' 

^ ' ' 1 V — 9 has a local minimum at x on U IH [0, l] d . 

The asymmetry between the two conditions in (14.291) is standard in the theory of 
optimal control. The first line says that V is a viscosity subsolution of the Hamilton- 
Jacobi equation in (0, l) d , the second one that V is a bilateral super solution. Generally 
speaking, V is also a bilateral subsolution at x G (0, l) d , i.e. H(x, V9(x)) = if V — 9 
has a local maximum at x, if there exists an optimal path reaching x. We refer the reader 
to [HJ §2.3, Chapter III] for more details. 

The boundary condition (14.301) is a boundary condition of Neumann type. This Neu- 
mann condition expresses the reflected structure of the controlled dynamical system. The 
viscosity formulation of the Neumann boundary condition has been introduced in [24j . 
In what follows, we will explain the link between this weak formulation and the standard 
Neumann condition. 

□ Proof. The proof is standard. We first give a suitable version of the Bellman dynamic 
programming principle for the quasi-potential V. Then, we will deduce Proposition 14.121 

Lemma 4.13 For all x G [0, l] d , for all t > 0, 

V{x) = inf {V{y) + [ L(fa,^ s )ds; (y, 0, xj)) : O = V, fa = x, = *(V)}- (4.31) 
Jo 

(In the above formula, we can assume that \if) 8 \ < 1 for a.e. s G [0,t] since L(x,v) = +oo 
for \v\ > 1. In particular, we can assume that \<j> 8 \ < 1 and \ii) 8 — <j> 8 \ < 1 for a.e. s G [0,£]. 
Indeed, |-?/> s | 2 = \ fa\ 2 + \ip s — fa\ 2 for a.e. s G [0,t].) 
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The proof of Lemma 14.131 is left to the reader. Details may be found in [5J Proposition 
2.5, Chapter III]. 

With the Bellman dynamic programming principle at hand, it is standard to prove that 
V is both a subsolution and a supersolution at x G (0, l) d , i.e. H (x, V0(x)) < if V — 9 
has a local maximum at x and H(x, V9(x)) > if V — # has a local minimum at x. (See 
for example the proof of [5| Proposition 2.8, Chapter III].) 

We now investigate the first boundary condition. 

For a given x G <9[0, l] d , we assume that there exists a continuously differentiable 
function 9 on U f) [0, l] d , £7 being a neighborhood of x, such that V — 9 has a local 
minimum at x on Z7 n [0, l] d . Without loss of generality, we can assume that 9{x) = V(x) 
and that the minimum is global on U fl [0, l] d so that V(y)—9(y) > for all y G Z7 H [0, l] d . 
We also assume (V0(x),n) > for all n G M{x). 

For t small, we can assume that y G £7 in the dynamic programming principle. We 
deduce that, for all t small, 

9(x) > inf {8{y) + / L(0 s ,^ s )cfe}, 



the infimum being taken over the same triples as above. Developing 9{x) — 9(y), we can 
write 

sup{ f (V9(<f) s ),ip s }ds- [ (V6{<t> s )^ s -j) s )ds- [ L(<j) S} ip s )ds; 

Jo Jo Jo l 4 -<J^J 

<p = y, <i>t = x, = ^(^)} >o. 

Having in mind that tp s - c/) s G R+Af(<p a ) (with R + Af(<p a ) = {0} if <p s G (0, l) d ) and 
\tps — <fis\ < 1 for a.e. s G [0, t], we deduce 

sup{ / sup max(O,-(V0(0 s ),n)) + / H(<f> a ,V9(<f> a ))da] 
Jo neN(<f> 3 ) Jo 

<p : [0,t] -> [0, l] d , t = rr, \<p s \ < 1 for a.e. s} > 0, 

Despite the lack of regularity of the boundary of [0, l] d , we can prove that, for \z — x\ 
small enough, N{z) C M{x). Since V6 1 and H are continuous, we deduce 

sup max (0,-(V9(x),n))+H(x,V9(x))+e t > 0, 

n£j\f(x) 

with e t — > as t tends to 0. By assumption, the first term in the above left-hand side is 
zero. This completes the proof. 

We now prove that V is a bilateral supersolution in (0, l) d and satisfies the second 
boundary condition. The idea follows from [5], §2.3, Chapter III] and consists in reversing 
the dynamic programming principle. This permits to write x as the initial condition of 
the controlled trajectory <fi. 

We let the reader check that for all x G [0, l] d and for all t > 0, 

V(x) > sup{V(y) - / L((f) s ,ip s )ds; (y,0,V) = 0o = x, fa = y, = tt(V)}. (4.33) 
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(Pay attention: there is no equality in (I4.33j) at this stage of the paper. Equality holds 
if there exists an optimal path from to x. This is the reason why we are not able to 
prove that V is a bilateral subsolution of the Hamilton- Jacobi equation.) 

Following [5J Proposition 2.8], this shows that V is a bilateral supersolution of the 
Hamilton- Jacobi equation in (0, l) d . 

We now prove the second boundary condition. As above, we assume that there exists 
a continuously differentiable function 9 on U IH [0, l] d , U being a neighborhood of x, 
such that 9(x) = V(x) and V(y) — 9(y) > for all y G U PI [0, l] d . We also assume 
(V0(x), n) < for all n G A/"(x). 

We choose a control with a constant speed. For a G R d , we choose ip s = x + as for 
all s G [0,t]. We then define = ^(VO- By (12. 5p . we can write 4> s = x + as — k s , with 
k s G E + jV(0 s ). For t small enough, t is in U and (14.331) yields 

9{x)>9{<p t )- [ L(<p s ,a)ds. 
Jo 

Developing 9((p t ) — 9(x) as in (I4.32p . we obtain 

/ (V9((J) S ), a)ds — \a\ I sup max(0, (V9((f) s ), n))ds — I L(<fi s , a)ds < 0. 
Jo Jo neM(<j> s ) Jo 

As above, we obtain 

(V9(x),a) — L(x,a) — \a\ sup max(0, (V9(x), n)) < 0. 

By assumption, sup^^^ max(0, (V0(x),n)) = 0. We deduce H(x, V9(x)) < 0. ■ 

We now explain the form of the equation when the quasi-potential is continuously 
differentiable on [0, l] d \ {0}. (We exclude from the set of differentiable points because 
there is a boundary condition of Dirichlet type in 0: V(0) = 0. Anyhow, as seen in the 
next section, there are specific examples in which V is continuously differentiable on the 
whole [0, l] d .) To this end, we introduce a modification of the gradient at the boundary. 
Assuming that exists at x G <9[0, l] d \ {0}, we set 

( [dV/dxi](x) if < Xi < 1, 
Vz G {l,...,d}, (y + V{x)).= < mm(\dV/dxi](x),0) if Xi = 0, 

[ m&x(JdV/dxi\ (x), 0) if Xj = 1. 

Similar modifications of the gradient of the quasi-potential appear in [2H Section II]. 
Following the notations introduced there in, we give another writing for V + V(x). We 
denote by VtV(x) the tangential part of W(x), i.e. 

Vm±AT(x), (V t V(x),u) = {VV(x),u), VneAf(x), (V T V(x), n) = 0. 

We also denote by e(x) the set M{x) H V, so that e(x) is an orthonormal basis of the 
cone generated by Af(x). (It satisfies (e, n) > for all e G e(x) and n G N(x).) Then, 
W{x) may be expressed as 



W(x) = V T V(x) + (Vy(x),e)e+ ^ (VV(x),e)e, (4.34) 

eSe+(x) ege_(x) 
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with e+(x) = {e G e(x), (W(x),e) > 0} and e_(x) = {e G e, (V7(i),e) < 0}. (In 
what follows, we will also make use of e (x) = {e G e(x), (VV(x),e) = 0}.) With these 
notations at hand, we have 

V+V(x) = V T V(x) + (VV(ar),e)e. (4.35) 

eGe+(a;) 

The above expression justifies the notation V+V(x). We are now ready to state: 

Proposition 4.14 Assume (A. 1—3). If the quasi-potential V is continuously differen- 
tiable on [0, l] d \ {0}, then it satisfies 

Vx G (0, l) d , H(x, VV(x)) = 0, (4.36) 

with the boundary condition 

Vx G 9[0, l] d \ {0}, H(x, V+V(x)) = 0. (4.37) 

By continuity of VV, we notice that (14.361) holds for all x G [0, l] d \ {0}. Moreover, 
we emphasize that (I4.37P is a boundary condition of Neumann type. If VV satisfies 
the standard Neumann condition, i.e. (VV(x),n) = for all n G N(x), at some x G 
d[0, l] d \{0}, then V+V(x) and VV(x) are equal. In this case, (OTP follows from fTOBD . 

As explained in (21J Section II], Hamilton- Jacobi equations under the standard Neu- 
mann condition, i.e. (VV(x),n) = for all n G Af(x) and x G d[0, l] d \ {0}, may not 
be well-posed. This explains why a weaker formulation of the boundary condition may 
be necessary. Anyhow, (14.371) is slightly different from the Neumann condition given in 
[2U Section II] since the original formulations in terms of viscosity solutions are different. 
(The optimal control problems are a bit different.) Moreover, the existence of "angles" 
along the hypercube [0, l] d induces additional difficulties in our framework. (In compar- 
ison, the boundary is assumed to be smooth in [24"1 Section II].) 

□ Proof. The proof is obvious inside the domain. (Choose 9 = V in the statement.) 

To prove the boundary condition, we characterize the continuously differentiable func- 
tions 9 such that V — 9 has a local minimum at x G d[0, l] d \ {0}. Following the proof 
of [3 Lemma 1.7, Chapter II], for a given p G M. d , there exists a continuously differ- 
entiable function 9 (on a neighborhood of x) such that V — 9 has a local minimum at 
x and V6>(x) = p if and only if the tangential part px of p is equal to V^V(x) and 
(p,n) > (W(x),n) for all n G J\f(x). 

In what follows, the typical value of p is p = V+V(x). Indeed, (V+V(x))r = VtV(x) 
and (V + V(x),n) > (W(x),ri) for all n G J\f(x). 

If (VV{x),n) < for all n G A/"(x), then (V + V(x),n) = for all n G H{x). (See 
(I4.35p .) Hence, we can apply both conditions in (14.301) . We deduce that 

H(x,V+V(x)) =0. (4.38) 

On the contrary, if (VV(x),n) > for all n G Af(x), the result is obvious. Indeed, 
V+V(x) = VV(x) in this case. Since VV is continuous, the Hamilton- Jacobi equation 
(I4.37P is true up to the boundary. 
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The intermediate cases may be treated by a similar argument of continuity. With the 
expressions (14. 34ft and ( 14. 35ft at hand, we set n+(x) = 2^ e ee + (x)ue (a;) e anc ^' ^ or £ > ^> 
y £ = x — en + (x). For e small enough, we have N{y e ) = e_(x) (if e_(x) is empty then 
y e G (0, l) d and M{y e ) is also empty) and, by continuity of W, (W(y e ),e) < for all 
e G e_(a;). By ll4~38l) . 

iy(y £ ,v + y(y £ )) = o. 

As e tends 0, V + V(y e ) tends to V + V(ar). Indeed, by (Q4]l and (T4T35]) . 

v + yy = vyy- ^ (v%),e) e = v%)- J2 ( w (^)> e > e 

eee_(y e ) eee_(i) 

— ►W(z)- V (VF(x),e)e = V+y(x). 

e^O ^ — ' 

e£e_ (a;) 

This completes the proof. ■ 

The boundary conditions are not formulated in a complete way in Proposition 14.141 
As stated below, (I4.30p implies additional conditions on the derivatives V a H(x, W(x)) 
and V a H(x, V+V(x)). In pH Section II], these additional conditions are formulated in a 
different way: the formulation used there in is about the signs of H(x, W(i) + An) and 
H(x,^ +V{x) + \n) for A 6 1 and n e M{x). We let the reader see how to pass from 
one formulation to another. Our formulation will be more convenient for the sequel of 
the paper. 



Proposition 4.15 Under the assumptions of Proposition for all x G d[0, l] d \ {0}, 
Ve G e+(x) Ue (x), (V a H(x,VV{x)),e) > and (V a H(x,V+V(x)) ,e) > 0, (4.39) 
and, 

Ve G e_(x), (V a H(x,VV{x)),e) < and (V a H(x,V + V(x)),e) > 0. (4.40) 

□ Proof. We fix x G d[0, l} d \ {0}. We start by proving that (V a H(x,V + V(x)),e) > 
for all e G e(x). 

We know that p = V + V(x) satisfies px = VtV(x), (p,n) > (VV(x),n) and (p, n) > 
for all n G N(x). For e G e(x) and A > 0, the same is true when replacing V+V(x) 
by S/+V(x) + Ae. (Indeed, (e, n) > 0.) According to the discussion led in the proof 
of Proposition 14.141 we can find an admissible 9 such that V8(x) = W + V(x) + Ae in 
(l4~30l) . We deduce that H(x,V+V(x) + Ae) > 0. Since H(x,V+V(x)) = 0, we obtain 
(V a H(x,V + V(x)),e) >0. 

As a by-product, the first inequality in (I4.39P is true when (VV(x),n) > for all 
n G M{x), i.e. when e_(x) is empty. In this case, W(x) = V + V(x). 

We now prove the first inequality in 1 14. 40 p when (VV(x),n) < for all n G M{x), i.e. 
e+(x) = e (x) = 0. Then, for e G e_(x) and < A < | ( VV(a;) , e) | , p = VV(x) + Ae 
satisfies (W(x),n) < (p,n) < for all n G N(x). By (14.301) . we deduce H(x,W(x) + 
Ae) < 0. Since H(x,VV(x)) = 0, we obtain (V a H(x,VV(x)),e) < 0. 
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We finally prove the first inequalities in (14. 39p and (14.401) without the assumptions 
e_(x) = or e + (x) = e (x) = 0. For e G e(x) and s > 0, we set y £ = x -e Y. e 'ee(x),e'^e e ' ■ 
For e > small enough, Af(y e ) = e. If e G e_(x), then (VV(y e ), e) < for e small enough. 
By the above analysis, (V a H(y £ ,'VV(y E )),e) < 0. Letting e tend to zero, we deduce 
that (V a H(x, VV r (x)), e) < 0. If e G e + (a;) U e (a;), we know, by the above analysis, 
that (V a H(y £ ,V+V(y £ )),e) > 0. As e tends to 0, V+V{y £ ) -> W(x). (To prove it, 
it is sufficient to check that (V + V(y £ ),e) — > (VV(x),e) = 0. Since (V+V(j/ e ), e) = 
(VV(y e ), e)l{<vv(2/ e ),e)>o}> this is true.) In the limit, we obtain (V a H (x, VV(i)), e) > 0. 
■ 

Uniqueness of the Solution. The above results provide the typical form, both in 
the viscosity and in the classical senses, of the Hamilton- Jacobi equation satisfied by 
the quasi-potential. A practical question is to identify the quasi-potential with a known 
solution of the Hamilton- Jacobi equation. 

Generally speaking, we are not able to prove that there is a unique continuous viscosity 
solution u satisfying both u(0) = and (14.291) and (14.301) . By adapting the techniques 
exposed in [7], we can only prove, under additional assumptions on H, that there exists at 
most one bilateral subsolution u to the Hamilton- Jacobi equation inside (0, l) d satisfying 
at the same time w(0) = 0, (14.291) and (14.301) . (Recall that u is a bilateral subsolution 
at x G (0, l) d if H(x, V9(x)) = for any continuously different iable 9 such that u — 9 
has a local maximum at x.) We won't perform the proof in the paper since we do not 
whether the quasi-potential is a bilateral subsolution of the Hamilton- Jacobi equation 
inside (0, l) d . 

Indeed, as already explained, the only thing we know is: if there exists an optimal path 
from to x G (0, l) d , then the quasi-potential is a bilateral subsolution of the Hamilton- 
Jacobi equation. Proving the existence of optimal paths for general quasi-potentials may 
be very difficult. (See e.g. [5] §2.5, Chapter III].) 

Anyhow, if the quasi-potential is assumed to continuously different iable, finding optimal 
paths may be easier. (See e.g. [TH] for a general result concerning the non- reflected case.) 
For this reason, we feel simpler to provide a uniqueness result to the Hamilton- Jacobi 
equation, but just for classical solutions. More specifically, we provide below a uniqueness 
result in which we both identify the quasi-potential with a known classical solution of the 
Hamilton- Jacobi equation and build optimal paths as solutions of a suitable backward 
reflected differential equation. 

We start with the necessary form of the optimal paths, if exist. To this end, we extend 
V+V to the whole [0, l] d \ {0} by setting V+V(y) = VV(y) if y G (0, l) d . 

Proposition 4.16 Under (A. 1—3), assume that the quasi-potential V is continuously 
differentiable on [0, l] d \ {0}. Let x G [0, l] d \ {0} and ((pt)t<o be a path satisfying <po = x 
and lim^-oo ip t = and achieving the infimum in the definition ofV(x). Then, (ft)t<o 
is absolutely continuous and verifies the backward reflected differential equation 

(ft = V a H(ip t , V+V(<pt)) - kt a.e. on the set {t < : <p t ^ 0}, (4.41) 

k being as in (12.51) . i.e. k t G Af(<Pt) if ft G d[0, l] d and k t = otherwise, and satisfying 
the compatibility condition 

(k t , V+V(if t )) = a.e. on the set {t < : tp t ^ 0}. (4.42) 
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(We emphasize that {t < : <p t ^ 0} is an interval. Indeed, if <p t = for some t < 0, 
then ip s = for s < t.) 

□ Proof. We admit for the moment the following 

Lemma 4.17 For every compact subset k C [0, l] d \ {0}, there exists a constant c K > 
such that for all y G k and v G M. d , \v\ < 1, 

L(y,v) > (v,V + V(y)) + c K \V a H(y,V + V(y)) - v\ 2 . (4.43) 

We then consider a path (fa)t<o with fa = x ^ 0, lim^.oo fa = and J- 00 ,o(4 ) ) < +°° 
(so that, without loss of generality, \fa\ < 1 for a.e. t < 0). By f)3.10p . we can find a 
measurable mapping t G (— oo, 0] i— > (f3 t , n t) G R+ x N(fa) such that for a.e. t < 

L ref (0 t , = L(0 t , fa + /5 t n t ). (4.44) 

(In the above formula, (3 t = if fa G (0, l) d or t G <9[0, l] d and (0 t , n) < for all 
n G J\f(fa). We refer to the proof of Theorem 13 .41 for the measurability property. We also 
note that \(j> t + j3 t n t \ < 1 for a.e. t < since L(fa, fa + fit^t) < +oo.) 

For a given compact subset k C [0, l] d \ {0} containing x, we set T K = inf{T > : 
y?_ T t K }- Lemma HUT] and (031]) yield for a.e. t G [-T K ,0] 

L iei (fa,fa) > (fa + Ptn t ,V+V(fa)) + c K \V a H(fa,V+V(fa)) - (fa + &n t )| 2 . 

We let the reader check that, for i G {1, ■ ■ ■ ,d}, the Lebesgue measure of the set {t < 
: (fa)i G {0, 1}, (<pt)i 7^ 0} is zero. (Indeed, the path is a.e. differentiable.) Hence, 
(fa, V+V(fa)) = [d/dt](V((j>t)) for a.e. t < 0. We deduce that for a.e. t G [-T K ,0] 

£ ref (^,0*) > [d/dt](V(fa)) + (&n t , V + y(0 t )> + c K |V a F(0 t , V+V(fa)) - (fa + (3 t n t )\ 2 . 
We deduce that satisfies 

J-ooM > V(4- Tk ) + / L TC{ (fa, fa)dt 

>V(fa) + I (p t n t ,V+V(fa))dt (4.45) 
+ c K f \V a H(fa,V + V(fa)) - (fa + (3 t n t )\ 2 dt. 

Noting that (f3 t n t , V + V(fa)) > for all t < 0, we complete the proof. ■ 

□ Proof of Lemma 14. 171 For y G k, v G W 1 , |t>| < 1, and e G (—1, 1), 

L(y,v) > (v,V + V(y)-e[V a H(y,V + V(y)) - v]) 

- H (y, V+V(y) - e [V a H(y, V+V(y)) -v\). 
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By Proposition 14.141 we know that H(y, V+V(y)) = 0. Applying Taylor's formula, in 
zero, to the function 

e 6 1) _> H (y, V + V(y) - e [V a H (y, V+V(y)) -v]), 

we obtain 

L(y,v) > (v,V+V(y)-£[V a H(y,V + V(y)) -v]) 

+ (V a H(y, V+V(y)) , e [V a H (y, V+V(y)) - v] > 
- (C/2)e 2 \V a H(y,V + V(y)) - v\\ 

withC = sup{\Vl >a H(z,V+V(z) -r)[V a H(z,V + V(z)) - v})\; z G k, v G M. d , \v\ < 1, rj e 
[—1, 1]}. By the regularity of if and VV^, the constant C is finite. Hence, 

L(y,v) > (v,V + V(y)) + (s - (C/2)s 2 ) \ V a H(y, V+V(y)) - v\ 2 . 

Without loss of generality, we can assume that C > 1 and choose e = 1/C in the above 
formula. This completes the proof. ■ 

In light of Proposition 14. 16[ we understand that the boundary conditions in Proposition 
14.151 describe the shape of the optimal paths (if exist) at the boundary. 

In what follows, we explain more specifically what happens in dimension two. For 
example, we consider x on the boundary with xi — and x<i G (0,1). In this case 
e(x) = {-ei}. 

If [dV/dxi\{x) > 0, then (VV A (x),-ei) < and -e x G e_(x). By Proposition \ZJS\, 
we know that ('V a H(x,W(x)),-e 1 ) < 0, i.e. (V a H(x, VV(a;)), e x ) > 0. Assume to 
simplify that (V a H(x,W(x)),e 1 ) > 0. By continuity, (V a H(y, W(j/)), ex) > for 
y in a neighborhood of x. If there exists an optimal path (<£>i)t<o reaching x at t — 
0, we understand from f)4.4ip that (tpt)t<o has to hit the boundary before reaching x. 
(Otherwise, there exists e > such that ((pt)i > for t G \—e,0), so that (<ft)i — 
(V a H((p t , W(<pt)), ei) > 0, and, the path cannot reach x.) This is illustrated by Figure 
El below. 



)( x 
it 




Figure 2: Typical optimal path: \dV/dxi]{x) > and (V a H(x, W(x)), e x ) > 



Similarly, if [dV / dx\\{x) < 0, i.e. —e% G e + (x), we know from Proposition 14.151 
that {V a H(x,\/V(x)),e 1 ) < 0. We assume to simplify that (V a H(x,W(x)),ei) < 0. 



32 



COMETS, DELARUE AND SCHOTT 



For y in a neighborhood of x, (V a H(y, W(y)), ei) < 0. Since [dV/dxi](x) < 0, 
we also have [dV/dxi](y) < and thus V + V(y) = W{y) for y close to x. Thus, 
(V a H(y, V+V(y)), ei) < for ?/ in a neighborhood of x. Then, the first coordinate 
of <p t , i.e. (ipt)i, is non-increasing as t grows up to 0. In particular, if ip_ £ = for 
some small e > 0, the path remains on the boundary from time — e to time 0. In 
such a case, k t = (V a H(<p t , W(<pt)), ei)ei for a.e. t £ [—£,0] so that (V+V(y^)?^t) = 
[<9V/<9xi](y? t )(V Q if(y? t , W(y2 t )), ei) > 0. This violates the compatibility condition fT442l . 
We deduce that the optimal path cannot hit the boundary in a small neighborhood of x 
before reaching x. This is illustrated by Figure [3] below. 




Figure 3: Typical optimal path: [dV/dxi](x) < and (V a H(x, VV r (x)), e\) < 

The case where [dV/dx\](x) = leads to too many different possibilities to make a 
general comment. (Anyhow, an example is provided in the next section.) 

Proposition 14.161 shows that, if optimal paths exist, the reflected differential equation 
f)4.4ip is solvable. We emphasize that f)4.4ip is not a reflected differential equation of 
standard type since the boundary condition is given by the terminal value of the tra- 
jectory. In particular, solving (I4.4ip is more intricate than solving a standard Skorohod 
problem. As shown below, the boundary conditions (14. 39p and (14.401) play a crucial role 
in the solvability of the equation (14.41 j) . 

Proposition 4.18 Assume (A. 1—3) and that there exists a function W E C([0, l] d , M), 
continuously differentiable on [0, l] d \{0}, such that, for allx £ (0, l) d , H(x, VW(x)) = 0, 
for all x £ d[0,l} d \{0}, H(x,V+W(x)) = 0, and, for all e £ e(x), (V a H (x, V + W(x)), e) > 
0. 

In addition, assume that, for all x £ d[0, l] d \ {0} and for all e £ e^(x) = {e £ 
e(x), (V a H(x, V + W(x)), e) = 0}, there exists a neighborhood U of x such that the sign 
of ( VW, e) is constant on the intersection of U with the face orthogonal to e, i.e. either 

VyeUD d[0, l] d , e £ J\f(y) => (VW(y), e) < 0, (4.46) 

or, 

VyeUn d[0, l} d , e £ N{y) => (VW{y), e) > 0. (4.47) 

Then, for any x £ [0, l] d \ {0}, there exist an absolutely continuous path (ipt)t<o and a 
real i(ip) £ (0, +oo] such that ip t = for all t < —i((p) ifi(<p) < +oo, fo = x and 

(p t = V a H(cp t , V+W(tp t )) - kt, for a.e. t £ {-i{<p), 0], (4.48) 
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k being as in (12.51) . i.e. k t G J\f(<ft) if ft G d[0, l] d and k t = otherwise, and satisfying 
the compatibility condition 

(k t , V+W((p t )) = for a.e. t G (-i(<p), 0]. (4.49) 

(Above, V+W{x) = VW(x) for x G (0, l) d .) 

The additional conditions (14.461) and (14.471) permit to avoid degenerate situations in 
which the sign of (V^e) changes at x for some e G M{y). Having in mind Figures [2] 
and El this permits to determine, a priori, the shape of the optimal paths reaching x. 

We emphasize that no assumption is necessary on the sign of (V a H(x, VW(x)), e), 
e G e(x). (In fact, using the convexity of H, we could prove that all the inequalities in 
(I4.39P and (14.40}) hold for W under the assumptions of Proposition 14.181 ) 

□ Proof. It is sufficient to prove that, for all x G [0, l] d \ {0}, there exist a real e > and 
an absolutely continuous path (ft)-e<t<Q such that (p = x and (I4.48P and (I4.49P hold 
on [— e, 0]. (By concatenating the local solutions, we obtain a global solution. When the 
resulting path hits the origin, the concatenation procedure stops. In this case, i((p) is 
finite. If the path doesn't hit the origin, i((p) is infinite. In the next theorem, we will 
prove under additional assumptions on W that the path tends to as t tends to — oo if 
i((p) = +oo.) 

If x G (0, l) d , the proof is trivial. (It is sufficient to solve, locally, the backward 
differential equation 

tpt = V a H(<p t ,VW(<pt)), t<0, 

with the boundary condition ip = x. Since V a H is bounded by 1 and VW is continuous, 
this is possible.) 

If x G <9[0, l] d \ {0}, the idea still consists in solving a backward differential equation, 
without reflection, but along a face of the hypercube. 

We first specify the choice of the face. By (14.461) and (I4.47H . there exists a neighborhood 
U of x such that, for all e G e w (x), the sign of ( VW, e) is constant on the intersection 
of U with the face orthogonal to e. (If e G e(x) \ e^(x), this is trivial by continuity of 
VW. If e G eff{x), this follows from (14.461) and 04.471) .) We then consider the (largest) 
face / containing x and orthogonal to e.f{x), with 

e f (x) = {e G e(x) : G U n d[0, l} d , e G M(y) (VW(y), e) < 0}. 

We denote by S the dimension of /. We can find a subset F C {1, . . . , d}, the cardinal of 
F being equal to 5, such that the family (ej)j(zF is a basis of the plane generated by /. 
We then consider the system of differential equations 

(&)i = (V a H(<p t , ^+W( ( p t )),e j ), t < 0, j G F, 
(<Pt)j = 0, t<Q, j e{i,...,d}\F, 

with the boundary condition ip = x. 

A priori, this problem isn't well-posed, even in a small time duration. Indeed, ipt 
may leave the hypercube in a zero time so that V a H(ip t , V + W(ip t )) rnay not be defined. 
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(Recall that Xi may be or 1 for some % G F.) To obtain a well-posed problem, we 
consider the following version 

= (v«ff(nfo], v + w(n[^])), ei ), t < o, j e f, 

(&)i = 0, t<0, je{l,...,d}\F, 

with the same boundary condition as above, where II denotes the projection on the 
hypercube. In the above system, either ej or — ej belongs to &f(x) for j G {1, . . . , d} \ F. 
(That is Xj = or 1.) Since (ipt)j = for such j's, &f{x) C A/"(II[</? t ]). For t close to zero, 
U[<f t ] G U, so that (W(n[^]),e) < for e G e/(ac). As a by-product, (V+W(n[<pt]), e^) 
is equal to for j G" F. For j G F, either (I% t ])j G (0, 1) or (1%*])^ G {0, 1}. In the 
first case, {V + W (Jl[ip t ]) , ej) is equal to (VW(II[<^]), ej). In the second case, either ej or 
— ej is a normal vector at H[<pt] an d belongs to e(x) \ Gf(x), so that (V + W (Jl[(p t ]) , ej) 
is still equal to {VW(Jl[ip t ]),ej). We deduce that V + W / (n[^J) may be expressed as 
^ j . gF (VV4 7 (n[y9 t ]), ej)ej in the above system. Thus, the coefficients of the system are 
continuous in the neighborhood of the boundary condition, so that the problem admits 
a solution on some interval [—£, 0], £ > 0. 

We now show that we can get rid of EE, at least for e small enough. To do so, it 
is enough to prove that <p t belongs to [0, l] d , or, equivalently, that (ft)j G [0, 1] for 
j G {1, . . • ,d}. For j G" F, this is obvious since (ipt)j = 0. We thus assume j G F. If 
(n[<^])j > 0, then (ipt)j = (n[</?t]) 3 - > 0. If (n[y? t ])j = 0, then — ej is a normal vector 
to the hypercube at n[</? t ]. By the boundary conditions satisfied by W, this implies 
(V a H(U[(p t ], V + V4 7 (n[( / 9 i ])), — ej) > 0. In this case, (ipt)j < 0. As t decreases on [— e, 0], 
(<^t)j cannot go below 0. Similarly, it cannot go beyond 1. We deduce that, for e small 
enough, ( 14.501) holds true. 

We finally prove that (14.481) holds on [— e, 0]. We can always write 

up t = V a H{tp u V + W{<ptj) - (^aH(i Pt ,V + W(ip t )),e j )e j 

je{i,...,d}\F 

= V a H(<p t ,V + W(<pt))-kt, 

with 

k= Yl (VaH(<p t ,V + W(<p t )),ej)ej= £ (V a H(<f t ,V+W(<p t )),e)e. 

je{l,...,d}\F eee f {x) 

Since e f (x) C H{<p t ), (V a H((p t , V + W((f t )), e) > for all e G e f (x). We deduce that 
(h)-e<t<o (with k = 0) satisfies (12.51) . The compatibility condition is obviously true. ■ 

We are now in position to state an identification property for the quasi-potential. 

Theorem 4.19 In addition to (A. 1-4), assume that, for all x G [0, l] d \{0}, (x, f(x)) < 
0. Assume also that there exists a function W satisfying the conditions of Proposition 



4-18 such that W(0) = 0. Then W is equal to the quasi-potential and the infimum in 
the quasi-potential is attained at (ft)t<o given by Proposition \4-18\ (We show below that 
such a path satisfies lim^.oo ip t = 0). 
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In the proof, we use the following lemma (the proof is given in Appendix, see Subsection 

E2}. 

Lemma 4.20 Under (A. 1—3), for any x G [0, l] d , the mapping a G M. d i— > H(x,a) is 
strictly convex at 0, i.e. the matrix ([d 2 H/daidaj](x, 0))ije{i,...,d} ^ s positive definite. 

□ Proof of Theorem 14.191 We first prove that V > W. For a given x G [0, l] d \ {0}, 
we can consider a path (ipt)t<o from to x, i.e. lini£_ > _ 00 ^ = and ipo — such that 
J-oofli^) < V"(x) + 5 for some 5 > 0. Then, ip is absolutely continuous. For a.e. t < 
such that ^ G (0, we have 

L ref (^ t ) =HMt) > fa,VWfa))-Hty t ,VW(iJ>t)) = [d/dt](W(<iPt)), (4.52) 

since W satisfies the Hamilton- Jacobi equation. The same holds for t satisfying ip t G 
d[0, l} d \ {0} and (ip t ,n) < for all n G N(ipt). For t satisfying xp t G d[0, l] d \ {0} and 
3n G M(ipt) such that n) = 0, we claim 

L ref (^) = inf L(ip t ,ip t + Pn) 

f3>0,n€Af(<pt),n±ipt 

> inf . [(V+W(^),^ + /3n)-F(V t ,V + W(Vt))] 

/3>0,neAf(ipt),n±i}>t 

inf . [(V + W(^),^ + /3n)], 

/3>0,n€A/'W't)> rl - l -</'t 

by the boundary condition of the Hamilton- Jacobi equation. By definition of V+W, we 
have (V+W(ipt), n) > for all n G J\f(ip t ). Hence, for t satisfying ip t G <9[0, l] d \ {0} and 
3n G Af(ipt) such that (ipt, n) = 0, we have 

L^(ip t ,iPt)>(V + W(iP t ),iPt). (4.53) 

For every i G {1, . . . , d}, the Lebesgue measure of the set {t < : (^t)j G {0, 1}, (^)j ^ 
0} is zero. Hence, we can replace (V + W(ip t ), ipt) by (VW (ipt) , ipt) = [d/dt](W(ipt)) in 
the above inequality. By (I4.52p and (14.531) . we have 

L rei (i/j t} ip t ) > [d/dt}(W(ip t )) for a.e. t such that ip t ^ 0. 

Setting i(ip) = inf{T > 0, = 0} and integrating from —i(ip) to (i(ip) being possibly 
equal to +oo), we deduce that V(x) + 5 > W(x). Letting 5 tend to 0, we deduce that 
V(x) > W(x). 

We now prove that V < W. We consider consider the path ((pt)t<o given by Proposition 
EM Recall from PH Chapter 5, (1.5)] that L(y, V a H(y, v)) = (v, V a H(y, v)) - H(y, v) 
for all (y,v) G [0, l] d x M d . By the Hamilton- Jacobi equation satisfied by W and by the 
compatibility condition (14.491) . we obtain, for a.e. —i(f) < t < 0, 

L(<p t , <p t + k) = (V+W(<p t ), & + k) - H(<p t , V+W(tp t )) 
= (V + W(<p t ),<p t ) = [d/dt](W(<p t )), 
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the last equality following from the same observation as above: for every i G {1, . . . , d}, 
the Lebesgue measure of the set {t < : (<ft)i £ {0, 1}, (<ft)i 7^ 0} is zero. Hence, for 
any T > 0, T > i(<p), 

U e \^ t ^ t )dt< / L{ipt,Vt + k)dt <W(x)-W{<p- T ). 

■T J-T 

If z(<y?) < +oo, the proof is over by choosing T = Otherwise, we have to prove that 
is an accumulation point of the path (ipt)t<o- 

Assume for a while that there exists e > such that, for all t < 0, \<pt\ > s. 
(In particular, i((p) = +oo.) By assumption, we know that, for all z G [0, l] d \ {0}, 
(z,V a H(z,0)) = (zj(z)) < 0. (Recall that f (z) = V a H(z,0).) By continuity of V a H, 
we can find a real rj > such that 

inf { (z, V a H(z, v)); z G [0, if, |z| > e, f G R d , |u| < f?} < 0. (4.55) 

Moreover, it is plain to see that for a.e. t < 

[d/dt] [|^| 2 ] = 2(<p t , V a H(<p t , V+W(<p t ))) - 2(<p t , k) 
<2(<p t ,V a H{<p t ,V + W(<pt))). 

(Indeed, if ((p t )i < 1, then {<pt)i(h)i = 0, and, if ((p t )i = 1, then (^)j > 0.) By (14.551). we 
deduce that there exists a constant c G (0, 1) such that 

- [d/dt] [\if t \ 2 ] > cl{\s7 + w{ Vt )\<v} - c~ l l{\v + w(tp t )\>v}- ( 4 - 56 ) 
By ( 14341 . for a.e. t < 0, 

[rfM][H/(^)] = L{<p t ,V a H(<pt,V+W(<pt))). 

By the strict convexity of L, for all z G [0, l] d \ {0}, L{z, V a H{z, V+W(z))) = if and 
only if V a H(z, V+W{z)) = f(z) = V a H(z,0). By the strict convexity of H(z,-) at 0, 
this is equivalent to V + W(z) = 0. We deduce that 

M{L(z,V a H(z,V + W(z))); z G [0, l] d , \z\ > e, \V+W(z)\ > r]} > 0, 

if not empty (i.e. 3z G [0, l] d , \z\ > e, \V + W(z)\ > rj). Up to a modification of c, we 
have 

[d/dt] [W(ip t )] > cl {[v+w{ipt)l>r]} . (4.57) 

We deduce that \{t < : \V+W((p t )\ > v}\ < +°°- Hence, \{t < : |V+W(p t )| < 7?}| = 
+oo. By (14.561) . there is a contradiction. We deduce that is an accumulation point of 
((pt)t<o- Hence, W(x) > V(x) so that W(x) = V(x). 

Actually, we can prove that lim t ^_ 00 {p t = 0. Indeed, by (14.541) . (W(<p t ))t<o is nonde- 
creasing (and bounded). We deduce that limt^.oo W((pt) = since is an accumulation 
point of the sequence (ft)t<o- Hence, every accumulation point a of the sequence (<pt)t<o 
satisfies W(a) = 0. Assume that there exists another accumulation point o ^ 0. Since 
is an accumulation point, we can find two decreasing sequences (£ n ) n >o and (s n ) n > , 
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converging to — oo, such that t n+ i < s n+ i < t n < s n for all n > 0, \<ft„\ = M/2 for all 
n > 0, \<f r \ > \a\/2 for all r G [t n , s n ] and n > 0, and \ f Sn — a\ — > 0. By (14. 56f) and (14.571) . 
we can find some constant C > (depending on a) such that t \— > — \ip t \ 2 + CW(ip t ) is non- 
decreasing on each [t n , s n ], n > 0. Hence, — |a| 2 /4 + CW(ipt n ) < — \a — y? s J 2 + CW(<p Sn ). 
Letting n tend to +oo, we obtain a contradiction. ■ 



5 Two-Stacks Model 

In this section, we consider a special case. It is a generalization of an interesting example 
introduced by Maier [22]. With d = 2, E = {1,2} and x = (x u x 2 ), let 

{\k[l - gx(x x )] , v = ei 

|(l-Z i )[l + ^ 2 (x 2 )] , v = -e 2 

with some ij G (0, 1) for all i & E, and some Lipschitz continuous functions g\, g 2 : [0, 1] — > 
[0,1), ^(z) > for ^ > 0. 

When c/i = <? 2 and Zi = 1/2, i G i£, this example reduces to that of Maier (see (4) in 
[29]). Here, the random environment £ governs the probability for each coordinate to 
jump, but not the jump distribution itself. Our treatment below is quite different from 
[2"§] . being more direct and leading to more general results. 

From (12.41) we compute 

^-((i-ti))' '=?>«■ 

In this example, all the assumptions (A. 1—4) are satisfied. The assumption of Theorem 
14.91 holds if P 2 is irreducible and g'i,g 2 > k' for some constant k' > 0. 

If both <7i(0) and g" 2 (0) are equal to zero, then /(0) = and the reflected differential 
equation (12.71) is simply the ordinary differential equation inside G. In this case, the 
hitting time of the stable equilibrium is infinite. If, on contrary, gj(0) > for some 
j G {1,2}, then the solution to the RDE (12.71) feels the reflection when hitting the j-th 
axis. After hitting the boundary, it moves towards the origin along the j-th axis. 

The function H can be expressed in terms of 

Hj(xj, atj) = In [cosher — 9j(xj) sinhaj] , j — 1, 2. 
From (13.4p . H(x, a) is the logarithm of the largest eigenvalue of the matrix 

Q(x,a) = [P(i,j){ke H ^^ + (1 -k)e H ^' a ^}] i;jeE . (5.59) 

Recall that E = {1,2}. By solving the characteristic equation, we find, with shorthand 
notations P(i,j) = Py, 

H(x, a) = In ^ (p n A + P^A 2 + \j (AiZ ~ ^2) 2 + 4P 12 P 21 A 1 A 2 ^ 



with 



Ai(x,a) = l ie Hi(^i) + (1 - / 1 ) e Ha(*2,o a ) ? 

A 2 (x,a) = l 2 e H ^ xl ' ai ^ + (1 - l 2 )e H ^ X2 ^ . 
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5.1 Identification of the Quasi-potential 

Although its expression does not look very explicit, the quasi-potential is quite simple. It 
can be guessed by observing that the discrete walk X n has an invariant measure, which 
obeys a large deviations principle: in view of [TBI Chapter 4, Theorem 4.3], the rate 
function - which is explicit here - should be the quasi-potential. 

In Maier's paper, the quasi-potential was identified by a Lagrangian approach and using 
the special structure of the separable Hamiltonian [291 p. 397]. Our approach here is an 
alternative yielding to a much shorter route for more general Hamiltonians. 

We start to look for the invariant measure. The Markov chain on {0, 1/m, . . . , 1} with 
nearest neighbor transitions (1/2) [1 =F gi(xi)} from x\ to X\ ± 1/m (pay attention to 
the change of sign between =p and ±) with reflection at and 1 has an invariant (even 
reversible) measure given for z = k/mby 

and vr! m) (0) = (l-^(0))/[2(l+^(0))] and vrj m) (m) = (1/2) ]lT=o^-9i{i))/{l+9l{i))- 
When the function gi is Lipschitz continuous, we obtain for large m and z G (0, 1), 



( m ) / \ 
tt{ [z) = exp 



= exp |— 2m J tanh l (g\{y))dy + o(m) \ (5.60) 

since tanh -1 (t) = (1/2) ln[(l + t)/(l —t)]. We define similarly, with g 2 instead of g±. 
The second observation is that the measure 

v( m \x,i) = 7r^V)4 m VVW (5-61) 

is invariant for our Markov chain (Xn/m, £„)„>o- Indeed, invariance of Trj; for the 
corresponding transition implies 

Wye [O.lfn^Z 2 ), Yl 4 m \xi)q{{x 1 ,y 2 ),i,{y 1 -x 1 )e 1 )=h4 m \yi)- 

xie{0,l/m,...,l} 

Hence, for all j G E and y G [0, l] 2 fl (m^Z 2 ), 

Y Y n[ m \x 1 )7ri m \x 2 )fi(i)P(i,j)q(x,i,y - x) 

i£E xe{0,l/m,...,l} 2 

= Y n^ n \x 1 )Trf , \x 2 )fJi{i)P(i,j)q(x,i,y-x) 

i£E \x 1 -y 1 \=l,x 2 =y2 or|a;2-2/2|=l,xi=yi 

= Y,P(ij)ii(i)[U + (i - h)]4 m) (yM m) (y2) 

= 4 m \ yi )4 m \y 2 Mj) 



DISTRIBUTED ALGORITHMS 39 

As a by product, the first marginal of v^ m \ i.e. v < ^ a \y) = ^i^iyij'^^iy'i) , is itself 
invariant for (X„ /m) n (which is not a Markov chain). From the relation (15.601) it is 
clear that this new measure satisfies a large deviations principle, with rate function 

W(x) = 2 / tanh- 1 ^))^ + 2 / tanh" 1 ^))^ (5.62) 
Jo Jo 

By P23 Chapter 4, Theorem 4.3], we then expect W to be the quasi-potential. By 
Proposition 14.191 we prove that this equality indeed holds. 

Theorem 5.1 The function W coincides with the quasi-potential. Moreover, for any 
point x G [0, l] d \ {0} ; there is one and only one optimal path (<ft)t<o from to x. The 
time reversed path (ifi-t)t>o is the unique solution to the reflected differential equation 
given by the law of large numbers (see Corollaru \3. 3\) . i.e. (ft = X-t f or o,llt <0. 

□ Proof. We check that all the assumptions of Proposition 14.191 are fulfilled. 

First Step. Hamilton- Jacobi Equation. The function W is clearly smooth. The gradient 
is given by 

VW(x) = 2(tanh- 1 (# 1 (a;i)),tanh- 1 (£ 2 (:E 2 ))) (5.63) 

On the boundary, V+W(x) = 2(0, tanh _1 (p 2 (^2))) for X\ = and x 2 £ (0, 1], V+W(x) = 
2(tanh~ 1 (^i(a;i)),0) for x x E (0,1] and x 2 = 0, V+W{x) = VW{x) for x ± = 1 and 
X2 G (0, 1] and for x\ G (0, 1] and x 2 = 1. For x = 0, we have V '+W{x) = 0. 
We recall the hyperbolic trigonometric identities 

tanha = ^-r , sinha = for a = 2tanh~ 1 (g) 

l + g 2 l-g 2 

For j = 1,2, the quantity exp(Hj(xj,aij)) = cosh(«j) — gj(xj) sinh(aj) is equal to 1 iff 
aij = or ctj = 2 tanh -1 gj(xj). From (15.591) . we deduce that Q(x,V+W(x)) = P for 
every x G [0, l] 2 . (With V + W(x) = VW(x) for x G (0, l) 2 .) Hence, the largest eigenvalue 
of Q(x,V + W(x)) is 1. We deduce that W satisfies the Hamilton- Jacobi equation (I4.36P 

-g3ZD. 

Second Step. Identification ofW. We first compute the gradient of H, with respect to 
a, in (x,V+W(x)), x G [0, l] d . Since, for all j G {1,2}, 

dHj sinh(a j) — gj(xj) cosh(aj) 



daj J ' Jy cosh(a j) — gj(xj) smh(aj) 

= gj(xj) for aj = [dW/dxj](x) 



we have 



?Q-(x,VW{x)) = [P(i,j)ligi{xi))ijeE, ^ X ^ W ^)) = [ p (hj)(l-k)92(x 2 )] ijjeE . 

(5.64) 

By simplicity of the top eigenvalue we know that H(x, •) is differentiate. For the same 
reason, the associated eigenvector v(x, a) is smooth in a. We thus differentiate the 
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equation Q(x, a)v(x, a) = exp(H(x,a))v(x,a) at a = WW(x). At such a point, Q = P, 



H = and v = 1 = (1, . . . , 1)*, so that 



VW(x))l + Pp-(x, VW{x)) = ^(x, VW{x))l + ^{x, WW(x)). (5.65) 
doti oat oai oa\ 

From (15.641) we have [dQ/dax](x, VW(x))l = g\(xi)\ with A = (li) i( zE, and by multiply- 
ing (15.651) by the invariant measure /i on the left, we get [dH/dai](x, VW(x)) = lg\(xi). 
With a similar computation for the partial derivative with respect to a 2 , we finally obtain 

V.iT(x, VW( X) ) = ( (1 l °f g j x2) ) = -/» . (5.66) 

Repeating the computations from (15.631) to (I5.66p . we have 

( ( ^ (0) ), forx 1 = 0, x 2 G(0,l], 

v a H(x,v + w(x)) = { }Zfrl{ 

[ ( J, for ^€(0,1], x 2 = 0. 



(5.67) 



It is plain to check that the assumptions of Proposition 14.181 are fulfilled. Therefore, W 
is the quasi-potential. 

Third Step. Optimal Paths. For a terminal value x G [0, l] 2 \ {0}, we have to prove 
that the time reversed path (x^)i<o satisfies (14.481) as well as (14.491) . It is sufficient to 
prove it locally: we prove that, for any x, (x-t)-e<t<o satisfies both (14.481) and (14.491) on 
a small interval [—£,0] for some e > 0. By (15.661) . this is easily checked if the terminal 
point x belongs to (0, l) 2 . If the terminal point x belongs to the boundary, several cases 
are to be considered. 

If x\ — and x 2 G (0, 1], the path (xt)t>o remains on {0} x [0, 1], so that (x,t)i = 
for t > 0. For some e > 0, we have (xt)^ £ (0, 1] for t e [0,e]. By (15.67!) . the second 
coordinate satisfies (Xt)z = ~~ (V a H(Xt> V+W(xt)), e 2) for t e [0, e]. Setting ip t = X-t 
for all t G [—£, 0] , we have 

(p t = V a H(<f t ,V + W{<f t )) - iUoK 

so that (ipt) £ <t<o satisfies (I4.48[) . Since (V+W(0, y), ei) = for all y G [0, 1], the compat- 
ibility condition is fulfilled. The same holds if x 2 = and X\ G (0, 1]. 

If x\ = 1 and x 2 G (0, 1], then the path (xt )t>o leaves the boundary immediately: for 
t > (and t small), Xt e (0, 1) 2 - Reversing the path, we conclude as above. The same 
holds if x 2 — 1 and x\ G (0, 1]. 

Fourth Step. Uniqueness of the Optimal Path. It remains to verify that the solutions 
to (14.481) are unique. Again, it is sufficient to prove that uniqueness holds locally for 
any starting point in [0, l] 2 \ {0}. If the starting point is in (0, l) 2 , this is obvious by 
time reversal. If x± = and x 2 G (0,1], we have (V a H(x, VW(x)), ei) = — /i(0) > 0. 
Assume for the moment that — /i(0) > 0. Then, by Figure [2} any solution ((pt)t>o to 
(I4.48[) touches the boundary before reaching x. Hence, there exists e > such that 
(<Pt)i = and (<fit)2 — —h{.( l Pt)2) for all t G [— £, 0]. Local uniqueness easily follows. 
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Assume now that /i(0) = 0. Then, for all y in the neighborhood of x, with y\ > 0, 
(V a H(y, VW(y)),ei) = —fi(yi) > 0. Again, any solution (<ft)t>o to (14.481) has to touch 
the boundary before reaching x (otherwise, it cannot reach the boundary) and we can 
repeat the argument. The same holds for x 2 = and x\ G (0, 1]. The case where X\ = 1 
and x 2 G (0, 1) corresponds (up to a symmetry) to Figure Eland local uniqueness is proved 
in a similar way. The cases where x 2 — 1 and x\ G (0, 1) and where {x\,x%) = (1, 1) are 
similar. ■ 



5.2 Deadlock Phenomenon for the Two- Stacks Model 

We discuss the deadlock phenomenon for the two-stacks model, that is for the domain G 
from (14. lip with i — 1. Our results should be compared to Section 5 in [29] . In view of 
Theorems 14.21 and 15.11 the set of exit points M. relates to the simple, one-dimensional, 
variational problem 

V = min{W(z, 1 - z); z G [0, 1]}. 

Then, x G M if and only if x = {z, 1 — z) with z minimizing the above problem. 
Observing that [d/dz](W(z, 1 — z)) — 2(tanh~ 1 (g 1 (,2)) — t&uhT 1 (g 2 (l — z))) has the same 
sign as gi{z) — c/ 2 (l — z), we distinguish a few remarkable different regimes (some of them 
being discussed in [29]) for the set A4 of deadlock configurations and for the shape of the 
optimal paths (which describe the typical course of a deadlock). 

Qualitative shape of optimal paths. For x = (xi,X2) G (0, l) 2 , we discuss the 
optimal path (ft)t<o from to x. By Theorem 15.11 ipt = X-t f° r all £ < 0. As long as 
the k-th coordinate {k = 1,2) of <p t is positive, it satisfies (<pt)k — ~fk{{. l Pt)k)- Hence, the 
time needed to make the fc-th coordinate move from to x k is 

f Xk dr 

tk = t k (x k ) = - TTT e (°> °°]- 
Jo fk(r) 

Note that t k is finite if the continuous function f k (or equivalently g k ) is non zero at 0, 
but t k is infinite if /^(O) = (since f k is Lipschitz continuous, \fu{ r )\ < Cr for r > in 
this case). In general, the duration of the instanton (p from to x is equal to maxjti, t2}- 

1. (Case A). gi(0) = #2(0) = 0. Then ip has an infinite duration. It never hits the 
boundary and does not feel the reflection. When g\ = g 2 , x\ = x 2 and A = 1/2, the 
optimal path is the line segment [x, 0]. But in general, the optimal path is not a 
line. 

2. (Case B). g\ (0) > 0,(72(0) > 0. Then, the optimal path has a finite duration. 
There is a smooth curve of points x's such that the reversed path from x to does 
not hit the axis (strictly) before 0: the curve is in fact defined by t\{x\) = t 2 (x 2 ). 
For x's such that ti(x\) < t 2 (x 2 ), \ x hits the vertical axis (strictly) above 0, and 
later on, moves down towards along this axis. 

3. (Case C). (?i(0) > 0,g 2 (0) = 0. For all x G (0, l) 2 , <p hits the vertical axis in a 
finite time, and later on, moves down towards along this axis reaching it in infinite 
time. 
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Some optimal paths are shown in Figures H] and [5] below. 



(Afli(**),(l-A)<to(l-**)) 




(z',l-z') 



Figure 4: Optimal deadlock point and path, Case 1 with I < 1/2, <?i(0) = g 2 {0) = 




Figure 5: Optimal deadlock point and path, Case 1 with gi(0) > 0,g 2 (0) > 0, and the 
exit point (z* ,1 — z*) in the general situation 



Some specific cases for the set M.. 

1. (Case 1). Assume that g\ and g 2 are strictly increasing on [0, 1]. Then X\ G [0, 1] i— > 
W(xi, 1 — Xi) is a strictly convex function so that M. reduces to a single point. If 
fi'i(O) > g 2 (1), then the function is increasing and the minimum is attained at X\ = 0, 
so that M. = {(0, 1)}. If <7i(l) < (?2(0), the function is decreasing and the minimum 
is attained at x\ = 1, so that M. = {(1,0)}. If ^i(0) < #2(1) and (72(0) < g'i(l), 
then the slope is negative at and positive at 1, so that M. = {{z*, 1 — z*)}, with 
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z* G (0, 1), the unique solution of gi(z*) = gi{\ — z*). This case is illustrated by 
Figures 0] and El 

2. (Case 2) Assume gi(z) = t? 2 (1 — z) for all z G [a, 6] (0 < a < 6 < 1), and #i 
[resp. (72] strictly increasing on [0, a] \J[b, 1] [resp. on [0, 1 — 6] |J[1 — a, 1]]. Then, 
gi(z)—g2(l — z) - as well as [d/dz\(W(z, 1 — z)) - is increasing [resp. zero, increasing] 
on the interval [0, a] [resp., [a, 6], [b, 1]]. Now, the set of minimizers is the interval, 

M. = segment [(a, 1 — a), (b, 1 — b)~\ 

as indicated in Figure El 




Figure 6: Optimal deadlock points, Case 2 



3. (Case 3) Assume gi(z) — g 2 (l — z) is negative on [0, a), positive on (a, c), negative 
on (c, b) and positive on (6, 1] (0 < a < c < b < 0). Then, W(z,l — z) is a 
double-wells, and the set of minimizers is a pair, 

M = {(a, 1- a), (6, 1-6)} 

see Figure [71 

By Theorem 15.11 there is a one-to-one correspondence between elements of Ai and 
optimal path (so-called instantons) to exit G. Therefore, there is a unique optimal path 
for the deadlock in Case 1, uncountably many in Case 2, and exactly two in Case 3. 

6 Limit Cycle 

In this section we work out an example where the system has, in the large scale limit 
m — > 00, a stable attractor, which is a limit cycle. Denote by 1 the vector (1, 1)*, and 
consider the differential system in R 2 , 



x t = h(x t ) 
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with 



h(x) 



-1 

1 



x 



1 

-1 

4 



64 x \x 11 

4 1 



x--l 



(6.68) 



Figure 8: An ordinary differential equation with a limit cycle 



whose phase portrait is given in Figure [HI The circle C centered at 1/4 with radius 1/8 is 
a stable limit set: trajectories spiral into it as time approaches infinity. More precisely, 
it can be checked that any point in [0, l] 2 is attracted by C. Moreover, the vector field 
on the axis is pointing inside the first quadrant, and, for I e [1, 2], the vector field on the 
sloping side \x\\ — I is pointing inside the domain G. 

Obviously, the reason for the existence of the limit cycle is that the vector field is the 
superposition of 

fci(*)=(j "q 1 )^"! 1 ) (6 ' 69) 
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- a rotation around the center (1/4)1 which preserves the norm of the vector {x— (1/4)1)— , 
and of 

h 2 (x) = ± [1 - 64 x \x - \l\ 2 ] [x - -l) (6.70) 

- whose effect is moving the system on the radius issued at the center towards the 
intersection of the radius and the circle C- . It is plain to check that the components of 
hi, ti2 are bounded on [0, l] 2 by a constant strictly smaller than 32. Let E = {1, 2}, and 
assume that /i is the Bernoulli law /z(l) = /z(2) = 1/2. With the components of 
/iGR 2 , define the transition by 

... J \(l ± ±hf\x)) for v = ±ei . , _ , 

p(x, i,v) — < ?) d * '( , z = l,2 (6.71) 

V ; \ ^li^/if^)) for i; = ±e 2 V ; 

Since /i is Bernoulli, the limit ordinary differential equation (13. 2ft is given here by 

with /i from ( 16.681) . Assumptions (A. 1—3) are fulfilled, as well as the counterpart to 
(A. 4) - with the attractor C replacing the stable fixed point 0. Most of the results 
of Section 0] can be generalized to this case, with the quasi-potential computed as the 
minimal action over all paths from C to the current point. For instance, (I4.13P becomes 

E Xm [r^} = exp[m(V + o(l))] 

for any sequence x m — > x G G, with 

V = inf{J o , T (0); 0o e C, = t, T > 0} . 

We cannot compute the exact value of the quasi-potential in this example, but it could 
be estimated numerically from above. Following [TOJ Chapter 5, Theorem 4.3], we could 
also provide a suitable version of Proposition 14.191 



7 Appendix A 

7.1 Proof of Lemma I^IOJ: successful coupling 

The proof relies on a tricky coupling argument. In [30], the authors investigate the large 
deviations for stochastic differential equations with a small noise: the coupling argument 
then follows from standard arguments for the Brownian motion. In our own setting, the 
standard stochastic analysis tools are useless and we need to construct a coupling for our 
purpose. 

Coupling. For an initial condition x G Z d , \x\± < Sm, the position of the walker is 
given by 

X n+1 = (2IT» - ld)(X n + f(X n /m,£ n ,U n )), 

with Xo = x. Here, f(z,i, •) denotes a function from (0, 1) to V such that f{z,i, U) has 
p(z,i,-) as distribution (typically, f(z,i,-) is an inverse of the cumulative distribution 
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function of p(z, i, •)). For another initial condition y G Z d , \y\i < Sm, the position of 
the walker can be defined in a similar way. The realizations (U n ) n >o may be the same. 
Nevertheless, the position may be defined with a different sample of uniform law. It may 
be also defined with the same sample but with a different function /. 

In what follows, we are seeking for a copy (X n ) n > of the walk, starting from y, such 
that X and X join up in a finite time. For this purpose, we assume \x — y\\ G 2N 
(otherwise, it is impossible). We will use the same sample of uniform law but a different 
function /. We thus write 

X n+1 = (2II< TO > - Id)(X n + f n {U n )), 

where f n is some random function from (0, 1) into V, depending on X n , X n and such 
that the conditional law of f n {U n ) with respect to (X n , X n , £ n ) is exactly p(X n , £ n , •). The 
explicit form of /„ has to be determined. 

To simplify, we will just denote (when possible) f n (U n ) by f n . Similarly, we will denote 
f(X n /m,£ n , U n ) by /„ (or f n (U n ) when necessary). 

Before providing an explicit form for f n , we investigate the L 1 -distance A n = \X n — X n \x- 
Loosely speaking, we want it to decrease with n. We thus compute A n +i in terms of A n . 
For this purpose, it is crucial to note that A n is always even (because of the particular 
choice for the initial conditions and for the reflection). We also recall the formula 

Va, feel, \a + b\ = \a\ + \b\ - 2(\a\ A \b\)l {ab<0} . 
If X n and X n are not on the boundary, we deduce 

d 

|A n+ i|i = |A„|i + \ f n - f n \i - 2^[|(A n )j| A \(f n )i - (/"■)*l] 1 {(A n ) i ((/„) i -(/, l )i)<0}- C^' 1 ) 

1=1 

If one of the two processes is on the boundary at time n, the difference |A n +i|i — |A n |i 
has the form |A n + g n — g n \i — |A n |i with g n and g n as in (12.31) . We can check that it is 
always bounded by |A„ + f n — f n \ \ — |A n |i. In other words, we can forget the reflection. 
To prove this assertion, it is sufficient to focus on each coordinate. If (X n )i = and 
{X n )i > 2, the proof is obvious. If (X n )i = and (X n )i = 1, the proof is the same except 
for (X n+ i)i = and (X n ) i+ % = 1. In this case, the processes switch. However, the result 
is still true. Other cases are treated in a similar way. Hence, in any case, (17. lft is true 
with < instead of =. 

Turn back to ( 17.11) . Again, |/ n — j n |i is always equal to 2, except for /„ = f n . To handle 
the last term, we introduce the following notations 

£"+(resp. E~, resp. E Q n ) = {u G V : (A n ,u) > (resp. < 0, resp. = 0)}. 

If f n = -f n G E~, the sum is equal to min(|(A n , / n )|, 2). If f n = f n or f n = -f n e 
E+ U E n , the sum is zero. If /„ J_ /„, \(f n )i - (f n )i\ is or 1 and (|(A n )i| A \(f n )i - 

(A) i |) 1 { (A„) I ((/„) l -(/, l ) l )<0} = \(fn)i - (/")ii 1 {(A„) i ((/„) i -(/ n ) t )<0} is als ° ° r 1: * is eC l Ual to 

1 if and only if f n G E~ and i is the coordinate of /„ or f n G E+ and i is the coordinate 
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of f n . Hence, 

|A n+ i|i < |A n |i + 2 - 21 {fn= j n} - 21 {fn± f n} (l {fneE - } + l { j neE + } ) 
-2[|(A n ,/ n )|A2]l {/n= _ /n!/n6S _ } . 

Noting that {f n 1 f n } is the complementary of {/„ = /„} U {/ n = -/„}, we have 
|A n+ i|i < |A n |i + 2 - 21 {fn= f n} - 2(l {/n6B - } + l { / ne£ + } ) 

+ 2 ( 1 {f n eE~} + 1 {/„££+}) t 1 ^/™} + !{/„=-/„}) 
-2[l(A„,/.)iA2]l {/n= _ /nj/n6S - } . 

We have {/„ = / n } = {/ n = f n G E+} U {/„ = f n e E~} U {/„ = /„ G Moreover 
{/n = ~fn e E-} = {f n = -f n G E+}. Hence, 

|A n+ i|i < |A n |! + 2 - 21 {fn= f neE o } - 2(l {/n6E - } + l { / neS + } ) 

+ 41 {/n = -/n,/nG^} ~ 2 [ I ( A « ' /« ) I A 2 ] 1 {/„ = -/ n ,/ n Gi^ } " 



Finally, 



|A n+ i|i < |A n |i + 21 {fneE + } 21 { f neE + } + 21 {f n eE°,fn^M 
+ 21 {f n =-f n j n eE-} 1 {\{A n ,f n )\=i}- 



(7.2) 



We claim that, for n < a = mi{k > : |A fc |i = 0} A inf{£; > : |A fe |i > 2[m5 1/2 \} 
(5 small enough), we can choose /„ such that {f n = -f n ,f n e E~,\(A n , f n )\ = 1} is 
empty and such that F{f n G E° n , f n ^ f n \ \F n } < J2 u eE° \p{ X n, £n, «) - P( X n, £„, u )] + , 
with T n = J***'*. 

The idea is the following. We define the random sets (i.e. they may depend on £ n , 
X n and X n ): A n (u) = {r G (0, 1) : f n (r) = u} and A n (u) = {r G (0, 1) : f n (r) = u} 
for u G V. The Lebesgue measures of these sets are known: |A n (w)| = p(X n ,£ n ,u) and 
|A n (w)| = p(X n ,^ n ,u). In the sequel, we just write p n (u) and p n {u) for these quantities. 

For each u G V, A n (u) is an interval (because of the construction by inversion of 
the cumulative distribution function). However, the geometry of A n (u) is free: we will 
perform the coupling by choosing the form of each A n {u) in a suitable way. Without loss 
of generality, we can assume that U u€ E°A n (u) is an interval with as left bound (see 
Figure ED • 

For u G E®, we can find a subinterval of A n (u) of length p n (u) A p n (u), with the same 
left bound as A n (u), and set for r in this interval f n (r) = u. Hence, f{f n — u, f n ^ 

Ul^n} < (Pn{u) -p n {u)) + , SO that 

nfn£ElJn^L\Fn}< £ \p(X n , £ n , u) - P (X n , £ n , u)} + . (7.3) 

u£E° 

This is exactly what we were seeking for. 

It remains to choose f n such that {/„ = —f n ,fn G E~, \ {A n , f n )\ = 1} is empty. For 
n < a, A n ^ and cannot be empty. Since |A n |j is always even, E£ cannot count one 
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single vector such that | (A n , u) | is odd. Hence, the set {u G V : | (A n , u) | G 2M+1} counts 
either zero element or more than two. If M = \E+\ = 1, the set {u G V : |(A„, u)\ G 
2N + 1} is empty and there is nothing to do. If M > 2, we can index E£ under the form 
= {v%, . . . , vm} with p n (vi) > p n ( v 2) > • • • > Pu{ v m)- Then, we can assume that the 
partition related to /„ is ordered as follows: 

Vm g El, A n {u) -< A n {-v x ) -< A n { Vl ) -< A n {-v 2 ) -< A n {v 2 ) < A n {-v M ) -< A n {v M ), 

where B\ -< B 2 means W(x,y) G B% X B 2 , x < y, B\ and B 2 being two subsets of [0, l] 2 
(see Figure M). 



E° 

n 



-Vi, fi 



-v 2 



V'2 



-V 3 , ^3 



Figure 9: Order for f n , M = 3. 



For /„, we already know that A n (u) intersects, for u G E®, A n (u) on an interval of 
length p n (u) f\p n {u). Then, we can complete A n {u), if necessary, that is if p n {u) > p n (u), 
so that U u£ E°A n (u) is an interval with zero as lower bound (see Figure fTUl). In particular, 
we have 

VueE° n , VvgE° n , A n (u)^A n (v). 
Then, we can complete the partition associated to f n as follows 

A n {v 2 ) -< A n {-v 2 ) -< A n (v 3 ) -< A n (-v 3 ) < A n {v M ) -< A n {-v M ) -< A n {-Vx) -< A n (v x ), 

see Figure [TUl 

.En . v 2 .~v 2 v 3 -v 3 -Vi Vi 



1 
Figure 10: Order for f n , M = 3. 



We now prove that, for 5 small enough and u G E~, the sets A n {u) and A n (— u) are 
disjoint. For 2 < i < M, the right boundary of A n {vi) is given by p n {E^) + p n {v 2 ) + 

Pn(-v 2 ) H \-p n (vi) and the left boundary of A n (-Vi) is given by p n (E%) + Pn(-vi) + 

Pn( v x) + " " ■ +Pn(—Vi-i) +p n (vi-i). By the Lipschitz property of p, the difference between 
p n (u) and p n (u) is bounded by (C/m)\X n — X n \x < 2C5 1 ^ 2 for every u G V. Since 
Pn(vi) > p n (vi), we have 

Pn(E°) +p n (-V 1 ) +p n (Vi) H hPn(-^-l) +Pn(Vi-l) 

>P„(Eq) +p n {-V 2 ) +p n (v 2 ) H \-Pn{-Vi-l) + Pn(Vi-l) + Pn{Vi) + Pn(~Vl) - ACd5 1/2 

> Pn(Eo) + Pn{-V 2 ) + p n (v 2 ) H h Pn(~^-l) + Pn{Vi-l) + Pn{Vi) +C- 4Cd5 1/2 , 
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with c = mf{p(z, i, v ); z G [0, l] d , i G £7, t> G V} > (see Assumption (A. 2)). For 5 
small enough, we obtain A n (vi) D A n (— Uj) = 0. It remains to prove the same thing for 
i = l. The right boundary of A n (— v\) is given by p n (E®) +p n (—vi) and the left boundary 

of A n {v x ) is given by p n {E° n ) + p n (v 2 ) + j3 n (-t> 2 ) H Vp n {v M ) +Pn{-y M ) + Pn(-vi) > 

p n (E®) + p n (— v\) + 2c — 4Cc?5 1 / 2 . This completes the construction of f n for M > 2. 

Hitting Time. Recall that T n = T^' x '^ for all n > 0. By (O) and (EJ), we have for 
5 small enough (say 5 < po for some po > 0) and n < a 

EOA^aM^J - |A n |! < 2( Pn - p n ) + (£°) + 2 Pn (£+) - 2p n (£+), (7.4) 

where (p n (-) — £>n(-)) + (^) = J2 u eA(Pn( u ) ~Pn{u)) + for any subset A of V (the same holds 
for Pn(A) and p n (A)). By (14.221) in Theorem 14.91 we have 

2(p n -p n ) + (E° n ) + 2p n (E+) - 2p n (E+) 
= 2( Pn -p n ) + (E° n ) +Pn(E+) -p n (E+) 

+ (1 - Pn (E° n ) - p n {E-)) - (1 -p n (E° n ) -p n (E-)) 
= 2( Pn - p n ) + (E° n ) - ( Pn - p n )(E° n ) + ( Pn - p n )(E+) - ( Pn - p n )(E-) 

= ^2 \Pn(u)-Pn(u)\+^2(p n (u)-Pn(u))sgn({A n ,u)) 
ueA,u±A n MgA 

< -(«;/m)|A n |i. 

By (E3|, we can write |A n+ i|i = \A n \ x + 2e n+1 , with e n G {-1,0, 1} and ¥,(2e n+1 \ F n ) < 
— (ft;/m)|A n |i for n < a. Hence, E(|A n+ i|i|.F n ) < (1 — K/m)\A n \i for n < a, so that 
((1 — K/m)~ nAf7 | A nAcr | 1 ) ra > is a super martingale. We deduce that, for all n > 1, 

2P{ct > n}(l - K/m)- n < E[(l - «;/m)" CTAn |A CTAn |i] < 25m. 

We obtain 

E(<r) < 5m ^(1 - K /m) n < CW, (7.5) 

n>0 

for some constant C" > 0. 

We now investigate P{|Ao-|i > 2[m5 1/2 J}. Since ((1 — n/m)~ nAa \ A nACT |i) n > is a super- 
martingale, we have for all n > m 3//2 , 

(1 - /c/mJ-^E^^a/ajlA^nlJ < E[(l - «/m) - CTAn I A ffAn | J < 25m. 

Letting n tend to +oo, we deduce (changing if necessary the value of C) 

(1 - K/m)- mi/2 ¥{a > m 3/2 , (A^ > 2[m5 1/2 \} < C. (7.6) 

It remains to see what happens for a < m 3//2 . We set q+ = P{& n = l|-7~n} and q~ = 
F{e n = — l\J- n }. Conditionally to the past, the process (|Afc|i)fc>o doesn't move at time 
n with probability 1 — (g+ + q~). Conditionally to moving, it jumps with probabilities 

In /(in + In) and In / (in + In)- SinCe H £ n+l\^n) < 0, We have ?+/(?+ + q~) < 1/2. 

Hence, the time needed by the chain (|A n |i) n > to reach 2|_W) 1 / 2 J is (stochastically) larger 
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than the time needed by the simple random walk to hit 2 [m^ 1 / 2 ] — 2 [m5\ when starting 
from zero. Hence, 

¥{a < m 3 / 2 , |A ff |! > 2[m5 1 / 2 \} < P{r 2Lm51/2J _ 2Lm5J < m 3 / 2 }, 

where denotes the hitting time, by the simple random walk, of a given integer L. It is 
well known (see e.g. [2H Chapter 10]) that P{r^ < m 3//2 } < exp(— aL + m 3 ^ 2 ln(cosh(a))) 
for any a > 0. Choosing a = m _3//4 , we have ¥{tl < m 3 / 2 } < exp(— Lm" 3 '' 4 + 
m 3 / 2 ln(cosh(m~ 3 / 4 ))). If L = 77m, for some 77 > 0, P{r^ m < m 3 / 2 } < C"exp(— 77m 1//4 ). 
Hence (changing C if necessary), 

P{a <m 3/2 ,|A (7 | 1 > 2[m5 1/2 \} < C exp(-2(5 1/2 - 5)m 1/4 ). (7.7) 

We can complete the proof of Lemma 14.101 We have, for all t > S, 

\F x {T {m) > m 2 t} - ¥ y {T {m) > m 2 t}\ < 2F{a > m 3 S or |A CT | ^ 0}. 

By dZSD, P{ff > m 3 S} < C'8S- x m- x . By (j£E) and (J7Z3), P{|A CT | ^ 0} < C"[exp(-2((5 1 / 2 - 
£)m 1//4 ) + exp(— /tm 1 / 2 )]. This completes the proof. 

7.2 Proof of Lemma 14^01 

For a given x G [0, l] d , we have to prove that the bilinear form V 2 a i^ : A G M. d 1— ► 
j AjAj[<9 2 if/<9aj<9a.,](x, 0) is positive definite. We first note that the bilinear form 

Sf : A G M d h- Et=i EfeeB )"(^)AiA i (E[(/ i / i )(j;, A;, f/)] - E[/i(x, A;, 17) ]![/,,• (z, A;, 17)]) in- 
duced by the averaged covariance matrix of the random vectors f(x,k,U) (U following 
the uniform distribution on (0, 1)) is nondegenerate. Indeed, for all A G M d , Jensen's 
inequality yields 



= E E /*(*) fc, £/)) - E /*(*) (E W<(*. *. f 7 ))) 2 

i=l fceB fce£ i=l 



i=l keE k£E i=l 

d d 



> E E torn*, *, ±«) - E E ^a? ^*'^"/^*'"^ 1 
=fv ^(fc) 4p(l,fc ;1 p(M r et) > 0, 

with p(x,k,±ei) = p(x,k,ei) + p(x, k, — e$). In what follows, we provide an explicit 
expression for V 2 Q i7 and then compare it to £/. We know that the leading eigenvalue of 
the matrix Q(x, a) (see ( 13. 4ft ) is simple and equal to exp(i7(x, a)). As a by-product, the 
coordinates of the corresponding eigenvector v(x, a) (i.e. of the i 1 normalized eigenvector 
with positive entries) are infinitely differentiable with respect to a. In particular, we can 



DISTRIBUTED ALGORITHMS 51 

differentiate twice the relationship Q(x, a)v(x, a) = exp(H(x,a))v(x,a) with respect to 
at, aij. We obtain 

[gt, + q£-] (x, a) = [(j£v+ g-) eMH)} (x, a) 

r <9 2 Q ^ dQ dv ^ dQ dv ^ ^ d 2 v -, 
ctajaj daidcxj dctj dai dctidcxj ' 

r, <9 2 ff + aff 9u_ + OH dH ^ + <9 2 t; + ch^dH_, eMH)] (x a) 

dcxidatj daidcxj dotidotj dotidaj dctidatj ' 

For a = 0, we know that Q(x, 0) = P (so that t> (x, 0) = 1 = (1, . . . , 1)') and [<9<3/<9aj](x, 0) = 
(P k}k ,E[fi(x, k, U)]) ktk i£E- Hence, for every k e E, E[fi(x, k, U)]+J2k'eE P kjk >[dv k r/dai](x, 0) : 
[dH/dai](x, 0) + [<9vfc/<9a!i](a;, 0). Integrating with respect to the invariant measure /i, we 
deduce that [dH/dai] (x, 0) = J^ fce£; /i(/c)E[/j(x, /c, f/)] = . Finally, 

J2 (W - P) fc , fc ^(x, 0) = E[fi(x, k, U)] - fi(x). (7.8) 
Applying the same method for the second order derivatives, we obtain for every k e E: 



E[(fJ 3 )(x, k, U)} + Pk,k'g-^(x, 0) 



+ [ E ifi& k > U)]P k ,v - fi(x)6 k}k ,] -^-(x, 0) 
+ ^ [nfj(x,k,U)]P k!k , - f 3 ( x )6 k , k ,]^(x,0) 



da, 

k'&E % 

d 2 H . . . d 2 v k 



x,0) + [fJ J }(x) + —^-(x,0). 



dctidaj dctidaj 



By (17. 8p . we have 



Kk) MMx, k, U)]P k , k , - fi(x)5 k>k ,] j*-(x, 0) 

k,k'EE i 

= y m[nfi(x,Ku)\-fi(x)]p k , k ^(x,o) 

k,k'eE i 

= y, m(^-p)k,k^(x,o)p k ,k^(x,o) 

k,k',k"&E % i 

= ((^-p)^-(x,o),p-^(xM^ 
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where (•, denotes the scalar product on L 2 (n) and Id the identity matrix on E. We 
now integrate ( 17. 9p with respect to the invariant measure, we deduce 

>H{ : (x,o) = Y,tik)n(fifj)(x,k,u)\ - [mix) 



% 3 keE 



+ «H-n£(«,0),p£(,,0)>, (™> 
+ ( (Id-P)^(,,0),P^fcO)>„. 



Using (17.81) . we deduce 



= ((Id-P)^,0),(Id-P)|^(,,0)>,. 



(7.11) 



Plugging ( 17. lip into ( 17.101) . we obtain 
d 2 H 



[x, 



()( , , . - 0) = S (x, fc, £/)] - ^ ^)ELA(x, fc, COM/, (x, fc, £/)] 

* 3 keE keE 

+ <( M -P)* (l ,0),(Id-P)Jl(*,0)>, 



(9f dv dv dv 

l _(,,0),P^-(x,0))„ + ((Id-P)^-(,,0),P^ 



For all (Ai)i<i< d e M d and k G £, we set w A (A;) = ^ a =1 A* (x, 0). Then, 

V*, a tf (A) = £/(A) + ((Id - P)u\ (I - P)u x ), + 2((Id - P)u\ Pu x ),. 
= S f (X) + (u\u x ) ll -(Pu\Pu x ) ll 

= £f(X)+ [ dfi(k)E k [(u x (^)) 2 ] - [ dpL(k)[E k u x (^)] 2 >£ f (X). 

J E JE 



This completes the proof. 
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